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PROTEST RELATING TO NEW CLAIMS UNDER 37 C.F.R. $ 1.291(c) 

This Protest is filed on behalf of the following companies: Amazon.com, Inc., Buena 
Vista Home Entertainment, Inc., DreamWorks LLC, Funai Electric Co., Ltd., LG Electronics 
Inc., Lions Gate Entertainment Inc., Metro-Goldwyn-Mayer Studios Inc., the Motion Picture 
Association of America (MPAA), Paramount Pictures Corporation, Pioneer Electronics (USA) 
Inc., Playboy Entertainment Group, Inc., Samsung Electronics Co., Ltd., Sanyo Electric Co., 
Ltd., Sharp Corporation, Sony Electronics Inc., Sony Pictures Entertainment Inc., Thomson, Inc., 
Toshiba America Information Systems, Inc., Twentieth Century Fox Film Corporation, Victor 
Company of Japan, Ltd., the Video Software Dealers Association (VSDA), Vivendi-Universal 
Entertainment LLLP, Wal-Mart Stores, Inc., and Warner Bros. Entertainment Inc. (the 
"Protestors"). 

This Protest follows the December 6, 2004 Amendment filed by applicant Multi-Format, 
Inc. ("Multi-Format") in connection with the above-identified reissue application. By the 
December 6, 2004 Amendment, Multi-Format cancelled all previously pending claims (26-255) 
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and added 38 new claims (256-293), all of which are substantially different from the cancelled 
claims. This Protest is directed to the newly-added claims and addresses issues that have 
particular application to those claims, including restriction acquiescence, recapture estoppel, 
written description, and the applicability of certain prior art references. Accordingly, this Protest 
"raises new issues which could not have been earlier presented," and is appropriate under 37 
C.F.R. § 1 .291(c). 1 In addition, this Protest discusses the relevance of newly-identified prior art 
to the new claims, providing a further basis for this submission. See 37 C.F.R. § 1.291(c) 
(providing that submission of "additional prior art" warrants further protest). 

INTRODUCTION 

By its December 6, 2004 Amendment, applicant Multi-Format cancelled all previously 
pending claims and added 38 new claims (256-293) in their place, with newly-added claims 256, 
276 and 285 being independent. New claim 256 is directed to an audio-visual method including 
the steps of receiving video and audio information, performing non-linear editing, and storing the 
edited video information. New claim 276 is directed to a system for capturing and recording 
digital video information including a video camera and a digital video recorder. New claim 285 
is a method claim corresponding to the system recited in claim 276. 

As discussed in detail below, all of these new claims, as well as the claims that depend 
from them, are barred by the doctrine of restriction acquiescence. The new claims are also 
barred by the doctrine of recapture estoppel. Because the issues of restriction acquiescence and 

1 In its December 22, 2004 "Response to Protest Under 37 C.F.R. § 1.291(a)" ("Response to Protest"), Multi-Format 
asserted that the December 6, 2004 Amendment moots the arguments made in the July 12, 2004 "Protest Under 37 
C.F.R. § 1 .291(a)" ("July 12, 2004 Protest"). On this point, Multi-Format contended that "the landscape of the 
pending claims was changed. Previously pending claims 26-255, to which the arguments of the protest were 
directed, were canceled, and new claims 256-293 were added. The language of the new claims differs from that of 
the previously pending claims, and none of the arguments of the protest are directed to the new claims . . .." 
(Response to Protest at 3.) Thus, Multi-Format has implicitly conceded that this Protest clearly "raises new issues 
which could not have been earlier presented" by the July 12, 2004 Protest. Applicant's new claims 256-293 
essentially present an entirely new application which justifies the filing of a new Protest. 
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recapture estoppel are specific to particular claims, the manner in which these doctrines apply to 
the newly-submitted claims could not possibly have been addressed in the July 12, 2004 Protest. 

The new claims are also unpatentable over prior art - both prior art previously of record 
and additional prior art not of record. The manner in which the prior art renders the claims 
unpatentable is discussed in detail below. Finally, new reissue claims 256-275 lack an adequate 
written description with respect to the common claim term "non-linear editing," and dependent 
claims 261-263 fail to meet the written description requirement for additional reasons, thereby 
providing independent bases for finding such claims unpatentable under 35 U.S.C. § 1 12, H 1 . 

None of the issues raised herein could have been raised before the new claims were 
presented. 

L NEW CLAIMS 256-293 ARE BARRED DUE TO THE DOCTRINE OF 
RESTRICTION ACQUIESCENCE 2 

New claims 256-293 are directed to inventions that were not elected in response to the 
restriction requirement made during prosecution of the original application that issued as U.S. 
Patent No. 5,537, 1 57 ("the 4 1 57 patent"). The new claims are not generic claims that link 
different species. Each of the new claims is directed to an invention that could not have been 
prosecuted following the election made during prosecution of the original * 1 57 patent. 
Accordingly, such claims cannot be prosecuted in this reissue application. 

A. Restriction Requirement and Voluntary Election in the Original Application 
In an April 10, 1995 Office Action during the prosecution of the '157 patent, the original 
Examiner issued a restriction requirement between two subcombinations: Group I (including 
application claims 1-20), characterized by the Examiner as being "drawn to a system for 
converting an image into multiple format," and Group II (including application claim 21), 

2 The legal standards associated with the doctrine of restriction acquiescence were set forth in the July 12, 2004 
Protest and will not be repeated here. 
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characterized by the Examiner as being "drawn to a video production system for generating an 
image." The Examiner stated that "Inventions I and II are related as subcombinations disclosed 
as usable together in a single combination." The Examiner then explained how the 
subcombinations are distinct from each other and why restriction was proper. (See generally 
April 10, 1995 Office Action at 8-9.) 

In response to the restriction requirement, the applicant for the '157 patent voluntarily 
elected to prosecute the claims of Group I, but failed to file a divisional application as to the non- 
elected claims. The application for the ' 157 patent was thus restricted to prosecution of claims 
directed to that one particular subcombination, i.e., a system for converting an image into 
multiple formats. After the restriction and acquiescence, no other subcombinations could have 
been prosecuted - whether in the original application or by reissue of the ' 157 patent that issued 
from the original application. 

B. None of the Newly Added Claims in the Present Application are Directed to 
the Originally Elected Subcombination and Therefore Cannot Be Allowed 

The newly added claims should be rejected due to lack of error supporting reissue 
because they are not directed to the subcombination that was voluntarily elected by applicant in 
the original application following the Examiner's restriction requirement - a system for 
converting an image into multiple formats. See Manual of Patent Examining Procedure (MPEP) 
§§ 806 and 806.05(d). 

The reissue application now includes three independent claims: claims 256, 276, and 
285. New independent claim 256 is directed to an audio-visual method that includes the steps of 
receiving video and audio information, performing non-linear editing, and storing the edited 
version of the video information on a high-capacity storage medium. In other words, new 
claim 256 is not directed to a system for converting an image into multiple formats. Also, no 
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claim that depends from new claim 256 is directed to a system for converting an image into 
multiple formats. New claim 256 and its dependent claims are directed to a completely different 
subcombination than the subcombination that was elected during prosecution of the original 6 157 
patent and cannot therefore be prosecuted in this reissue application. Once a particular 
subcombination is elected, an applicant has no right whatsoever to prosecute claims directed to 
other subcombinations. That is the very purpose of a restriction requirement. 

Similarly, new independent claims 276 and 285 (and their dependent claims) are not 
directed to a system for converting an image into multiple formats, i.e., the subcombination that 
was elected during prosecution of the original '157 patent in response to the restriction 
requirement. These claims are directed to a video camera and digital-video recorder and 
corresponding method, which are completely different from the originally elected 
subcombination. 

It is also noted that Multi-Format filed a "Supplemental Declaration for Reissue Patent 
Application" on March 11, 2005, indicating that the error to be corrected in the reissue is that the 
'157 patent did not include claims directed to "an audio visual method wherein editing is 
performed on video information in a digital format having a frame rate of substantially 24 frames 
per second (fps) in a non linear manner and the edited video information is then stored on a high 
capacity storage medium in a format of compressed digital video having a frame rate of 
substantially 24 fps[.]" This statement further demonstrates that the new claims are directed to 
an invention that is completely different from the elected subcombination (which was a system 
for converting an image into multiple formats). 
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The present scenario is controlled by In re Watkinson, 900 F.2d 230, 14 U.S.P.Q.2d 1407 

(Fed. Cir. 1990) and In re Weiler, 790 F.2d 1576, 229 U.S.P.Q. 673 (Fed. Cir. 1986). In Wetter, 

the Federal Circuit provided as follows: 

Significantly, Weiler accepted issuance of the '923 patent with its 
claims to a single elected invention. By acquiescing in the 
examiner's restriction requirement, and failing to file divisional 
applications on the subject matter of non-elected claims, Weiler 
foreclosed (because that was not error) his right to claim that 
subject matter. If it were not error to forego divisional applications 
on subject matter to which claims had been made in the original 
application, it cannot on the present record have been error to 
forego divisional applications on subject matter to which claims 
had never been made. 

790 F.2d at 1582. Thus, new claims 276 and 285 (and their dependent claims) cannot be 

prosecuted in the current reissue application because they are not directed to the subject matter to 

which the original application was voluntarily restricted. 

C. Applicants' Arguments Relating to the Protestors' Characterization of the 
Subject Matter of the Elected Claims are Both Misguided and Precluded 

In its Response to the July 12, 2004 Protest, Multi-Format accused the Protestors of 
attempting to mislead the Examiner "by mischaracterizing the subject matter of claims 1-21." 
(Response to Protest at 9, footnote 4.) However, this supposed "mischaracterization" originates 
not from the Protestors but from the Examiner himself: it is a word-for-word recitation of the 
characterization of claims 1-20 given by the Examiner in his restriction requirement in the April 
10, 1995 Office Action. If the applicant believed that the Examiner had incorrectly characterized 
the invention defined in claims 1-20 during prosecution of the original '157 patent, it was 
incumbent upon the applicant to dispute the characterization then . However, the applicant did no 
such thing. Rather, the applicant acquiesced in the restriction requirement, elected the 
subcombination of Group I, and failed to file any divisionals. Multi-Format can hardly accuse 
the Protestors of mischaracterizing the invention and attempting to mislead the Examiner here 
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when the Protestors did nothing more than adopt a verbatim recitation of the original Examiner's . 

restriction requirement which is already of record. 

Moreover, Multi-Format is precluded from challenging the propriety of the underlying 

restriction requirement at this time. After having acquiesced in the restriction requirement, 

having voluntarily elected the claims of the Group I subcombination, and having canceled the 

non-elected claims, applicant cannot now recharacterize the claims of the original application or 

ignore the original Examiner's characterization of the claims. This situation was dealt with 

directly in In re Watkinson: 

[W]e . . . hold that the failure to file a divisional application, 
regardless of the propriety of the underlining restriction 
requirement , is not an error correctable by reissue under 35 U.S. C. 
§251. 

900 F.2d at 231-32 (emphasis added). As explained by the Federal Circuit in Watkinson, after 

acquiescing in a restriction requirement and canceling the non-elected claim, the applicant lost 

the opportunity to challenge the propriety of the restriction requirement. Id at 233. 

D. Restriction Acquiescence Cannot be Avoided by Reference to Original 
Dependent Claims 

Multi-Format essentially seeks to circumvent the restriction to a particular 
subcombination by pointing the Examiner in the direction of dependent claims and asserting that 
an applicant is free to submit claims directed to the subject matter of the dependent claims by 
themselves, without including the subcombination that was originally elected . This cannot be 
permitted, as it would effectively eviscerate the original restriction requirement. Again, during 
prosecution of the original '157 patent, the applicant elected to prosecute claims directed to "a 
system for converting an image into multiple format" in several independent claims, and, 
therefore, can pursue by reissue only claims that include this same subcombination. 



Appl. No. 10/004,046 



7 



16716-6 



Multi-Format asserts that new reissue claims 256-293 are "broader claims drawn on the 
elected claims, not on the non-elected claims," and maintains that the new claims could have 
been included with the original application that contained the elected claims. (Response to 
Protest at 1 1.) Not only are Multi-Format's assertions incorrect, they are irrelevant. 

In purported support, Multi-Format provides an example of original claims "directed to 
digital cameras," referring to original claims 5 and 6. (Id at 12.) Essentially, Multi-Format is 
arguing that as long as it continues to include at least one limitation from any original claim, it 
could have prosecuted such claim in the original 6 157 patent. This is, of course, not the case. 
Multi-Format has dug down into a dependent claim, which adds another (non-elected) 
subcombination, and has then completely eliminated the elected subcombination that was in th e 
independent claims . In other words, all that is left are the elements of non-elected subject matter 
without the originally elected subcombination. By doing this, Multi-Format is no longer 
claiming the original subcombination that was elected, but has shifted over to claim a completely 
different subcombination. This is precisely the type of legally impermissible tactic prohibited by 
Watkinson and Weiler, 

The fact that dependent claims added limitations to additional subcombinations does not 
mean that Multi-Format is free to ignore the originally-elected subcombination set forth in the 
original independent claims of the 6 157 patent. The elected subcombination cannot be 
eliminated by reissue and substituted with several different subcombinations, as Multi-Format 
attempts to do here. 

On page 1 1 of its Response to Protest, Multi-Format maintains that the new reissue 
claims 256-293 are "not substantially similar to non-elected original claim 21." Again, this is 
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completely irrelevant and flips the required analysis on its head. As stated in Ex Parte 

Pagilagan, Appeal No. 2001-1752, 2002 Pat. App. LEXIS 198 (B.P.A.I. 2002): 

Appellant's focus on differences between the reissue claims and 
the non-elected claims misplaces the focus of the inquiry. Whether 
the subject matter presented in the reissue claims is patentably 
distinct from the subject matter of the non-elected claims is not the 
issue. The issue is whether Appellant had a right to claim the 
subject matter now claimed in the original patent. It cannot be 
"error" to fail to include claims to subject matter which Appellant 
had no right to include in the first place. There was no right to 
claim a process in the original patent because Appellant acquiesced 
in a restriction requirement that limited the claims to a 
copolyamide composition and articles. If the Applicant cannot 
claim the subject matter in the original application, he has no "right 
to claim" it in the reissue. 

Id at * 1 6-* 1 7. After the restriction and election, the applicant had no right to claim any non- 
elected subcombination in the original 4 157 patent, and Multi-Format has no right to claim it now 
in this reissue application. 

E. In re Doyle is Not Relevant to Restrictions Relating to Subcombinations 
Finally, Multi-Format's reliance upon In re Doyle, 293 F.3d 1355, 63 U.S.P.Q.2d 1161 
(Fed. Cir. 2002), in its effort to obtain claims to inventions other than the originally-elected 
subcombination, is entirely misplaced. The current reissue application implicates the issue of 
whether claims to different subcombinations can be presented after restriction and election of a 
single subcombination. In contrast, Doyle was concerned with the situation of a species 
restriction and the ability to prosecute a generic claim following a species restriction and 
election. Doyle simply recognizes that following restriction to a particular species, an applicant 
is free to prosecute a generic claim that links different species. (This practice is governed by 
37 C.F.R. § 1.146 and discussed in MPEP §§ 809.02 and 809.02(a).) The reason for this is clear: 
the generic claim still reads on the elected species. The Federal Circuit in Doyle concluded that a 
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generic or linking claim could be prosecuted in reissue because such claim could have been 

prosecuted in the original application even after the election of a single species. 

Contrary to the genus/species situation, once an election to a particular subcombination is 

made - as it was in the original '157 patent - the applicant has no right whatsoever to prosecute 

claims that are not directed to the elected subcombination. Rather, by acquiescing to a restriction 

to a particular subcombination, claims directed to any other subcombination can be presented 

only by way of a divisional application. In adding new reissue claims 256-293, Multi-Format 

baldly states, without any basis whatsoever, that all of the new claims could have been 

prosecuted in the original application following the restriction and election. (Response to Protest 

at 1 1 .) This simply is not true. Because the original application was restricted to one particular 

subcombination, the reissue application can present only claims directed to that subcombination. 

II. NEW CLAIMS 256-293 ARE BARRED UNDER THE DOCTRINE OF 
RECAPTURE ESTOPPEL 3 

In view of the cancellation of all of the previous claims by Multi-Format, and the 
submission of an entirely new set of claims, the new claims must be freshly analyzed to 
determine the applicability of the recapture doctrine. Upon review of the newly submitted 
claims in the context of the prosecution history of the original 4 1 57 patent, it becomes apparent 
that these new claims are also barred under the recapture doctrine. 

On page 16 of its Response to Protest, Multi-Format asserts that "the only limitation 
added to avoid the prior art was the inclusion of high-capacity video storage used to store video 
information in a particular format." (Emphasis added.) By this assertion, Multi-Format tries to 
avoid the fact that claim 1 of the 6 157 patent was specifically amended to state that the graphics 
processor converts the display format of the program into an intermediate production format. 

3 The legal standards associated with the doctrine of recapture estoppel were set forth in the July 12, 2004 Protest 
and will not be repeated here. 
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Multi-Format pretends that this amendment was made not for patentability purposes, but rather 

to provide antecedent basis for the function of the high-capacity storage means. Multi-Format's 

argument, however, is completely belied by the prosecution history of the ' 157 patent. Indeed, 

Multi-Format's assertion that the August 11, 1995 Amendment was not made for patentability 

purposes is belied by the contemporaneous remarks it made in connection with that amendment: 

Claims 1-4, 8-1 1 and 13-20 stand rejected under 35 U.S.C. 
§ 102(e) as being anticipated by Hailey, U.S. Patent No. 5,243,433. 
Claim 1 has been amended to include high-capacity video storage 
means and an aspect always present in independent claims 13 and 
17, that is, an internal production or 'working' format which may 
be stored and/or used as the basis for input to the interface units so 
as to create a standard/wide-screen or HDTV output formatted 
imagery. Hailey neither implies nor suggests the use of such an 
intermediate format , nor does Haley include any means for the 
storage of images, temporarily or permanently during conversion 
directly from an input format to an output format. 

(August 1 1, 1995 Amendment at 10; emphasis added.) 

Thus, the applicant explicitly distinguished U.S. Patent No. 5,243,433 to Hailey 
("Hailey") through its remarks that claim 1 was amended to include the use of an internal 
production format in addition to video storage means. Both limitations were clearly argued to 
distinguish over Hailey. Moreover, the argument regarding conversion to an intermediate format 
was directed to all of the independent claims. The prosecution history of the '157 patent could 
not be any clearer. The applicant specifically argued the limitation contained in each of the 
independent claims regarding conversion into an intermediate production format to distinguish 
over Hailey. 

Multi-Format attempts to minimize the impact of the August 11, 1995 Amendment and 
the applicant's argument regarding conversion into a production format by asserting that the 
amendment and argument were not the reason the claims were allowed by the Examiner. Multi- 
Format maintains that there was no surrender with respect to converting the input program to a 
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production format "because this limitation was not added during prosecution of any claim to 
overcome the rejection to Hailey." (Response to Protest at 18.) As discussed above, this 
statement is untrue. Moreover, the Examiner's reaction to the surrendering amendment made by 
the applicant is completely irrelevant. The only relevant inquiry is what the applicant 
surrendered in its efforts to distinguish over the prior art. In effect, Multi-Format is arguing that 
the surrender of subject matter during the original prosecution was not necessary, because the 
Examiner did not allow the claims based upon such surrender. Again, whether or not the 
surrender was necessary, or "successful," is irrelevant. See In re Clement, 131 F.3d 1464, 1471, 
45 U.S.P.Q.2d 1 161 (Fed. Cir. 1997) (" every time Clement amended his claims, he intentionally 
omitted or abandoned the claimed subject matter") (emphasis added). It is the applicant's 
surrender of subject matter in response to a rejection - not the Examiner's response (or lack 
thereof) to the surrender - that is relevant. 

As noted in Clement, the recapture rule "prevents a patentee from regaining through 
reissue the subject matter that he surrendered in an effort to obtain allowance of the original 
claims." Id at 1468 (emphasis added). Multi-Format most certainly surrendered claim coverage 
that did not include conversion into a production format through both amendment and argument. 
Nothing was ever done during the prosecution of the original 4 157 patent to undo that surrender. 

The surrender-generating limitation of conversion into a production format is completely 
absent from newly-submitted claims 256-293. Thus, recapture estoppel clearly applies. Multi- 
Format cannot simply eliminate a limitation that was specifically added and/or argued in an 
effort to obtain allowance of the claims. 

Multi-Format's reliance upon Ex Parte Eggert, 67 U.S.P.Q.2d 1716 (Bd. Pat. App. & 
Inter. 2003) (see Response to Protest at 19) is misplaced, because, as discussed above, the 
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limitation regarding conversion into a production format has been entirely omitted. Eggert only 

applies when a surrender limitation is not entirely omitted, but rather is recited in a broader form. 

Here, the new claims contain absolutely no recitation of conversion into a production format. 

III. NEW CLAIMS 256-275 ARE UNPATENTABLE UNDER 35 U.S.C. § 112, f 1 FOR 
FAILURE TO SATISFY THE WRITTEN DESCRIPTION REQUIREMENT 

A. The 4 157 Patent Does Not Provide Written Description Support for the 
Method of Independent Reissue Claim 256 

New independent claim 256 recites an "audiovisual method" comprising three separate 
steps, including "performing non-linear editing on the video information in its digital format to 
create an edited version of the video information," and "storing the edited version" in a 
"compressed" format, which implicitly requires performing the "non-linear editing" step before 
the "storing" step. December 6, 2004 Amendment (emphasis added). The claim term "non- 
linear editing" is common to independent claim 256 and its dependent claims 257-275. 
However, this claim term is not common in the '157 patent, as it makes only a solitary 
appearance in the more than 13 columns of text in the original specification, is absent from the 
figures of the '157 patent, and has never appeared in any of the hundreds of claims presented 
before the recent December 6, 2004 Amendment. The named inventors, however, made a 
throwaway reference to "non-linear-editing" in the original '157 patent, which Multi-Format 
now attempts to conflate into the basis for 20 new claims in violation of the written description 
requirement of 35 U.S.C. § 1124 1. 

1. The Term "Non-Linear Editing" was Merely Mentioned Once, in 
Passing, in the Original 4 157 Patent 

The '157 patent mentions "non-linear editing" only once in passing: 

In particular, techniques such as non-linear-editing , animation, and 
special -effects will benefit from the implementation of this system. 
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('157 patent, Col. 4, lines 56-58; emphasis added.) Other than that single reference, there is 
absolutely no discussion of "non-linear-editing" in the '157 patent. There is no explanation as to 
what constitutes a "non-linear-editing" technique, nor elucidation regarding any "benefit" that 
the "non-linear-editing" technique will allegedly realize from implementation of the system 
described in the '157 patent. Moreover, there is no disclosure of how to "perform[] non-linear 
editing on [] video information" nor is there any suggestion of the specific order of steps to be 
performed - "non-linear editing" before "storing" - in the method now claimed by Multi-Format 
more than a decade after the application for the ' 1 57 patent was filed. 

Multi-Format has taken advantage of a passing reference about a "benefit" and the 
passage of time to lay claim to technology that obviously was not contemplated by the named 
inventors at the legally critical juncture of when they applied for the ' 1 57 patent. If they really , 
had invented an "audiovisual method" in which "non-linear editing" was performed on video 
information before storage in a "compressed" format, the named inventors would have 
demonstrated that they possessed this technology by actually describing such a process in the 
' 1 57 specification. The vague throwaway reference in the original specification - that "non- 
linear editing" is one of many techniques which will "benefit" from the "invention" as originally 
claimed - does not say whether one should edit and then compress, or vice versa. 

2. The Performance of "Non-Linear Editing" Prior to Storage is Not 
Adequately Described in the '157 Patent 

The term "non-linear editing" was added to the claims by amendment. "Any such 
amendment or insertion must comply with all statutes and regulations." Kingsdown Med. 
Consultants, Ltd v. Hollisterlnc, 863 F.2d 867, 874, 9 U.S.P.Q.2d 1384 (Fed. Cir. 1988). Of 
course, included among the "statutes and regulations" that must be observed is the first 
paragraph of Section 1 12 which provides: 
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The specification shall contain a written description of the 
invention, and of the manner and process of making and using it, in 
such full, clear, concise, and exact terms as to enable any person 
skilled in the art to which it pertains, or with which it is most 
nearly connected, to make and use the same, and shall set forth the 
best mode contemplated by the inventor of carrying out his 
invention. 

35 U.S.C. § 1 12, If 1. This statutory provision contains three separate and independent 
conditions on patentability, respectively referred to as the "written description," "enablement" 
and "best mode" requirements. See Univ. of Rochester v. G.D. Searle & Co., 358 F.3d 916, 920- 
21, 69 U.S.P.Q.2d 1886 (Fed. Cir. 2004). Multi-Format's new claims 256-275 violate at least 
the first of these requirements. 

To satisfy the "written description" requirement, a patent applicant is required to "convey 
with reasonable clarity to those skilled in the art that, . as of the filing date sought, he or she was 
in possession of the invention." Vas-Cath, Inc. v. Mahurkar, 935 F.2d 1555, 1563-64, 19 
U.S.P.Q.2d 1111 (Fed. Cir. 1991). "An applicant complies with the written description 
requirement 'by describing the invention, with all its claimed limitations.'" Gentry Gallery, Inc. 
v. Berkline Corp., 134 F.3d 1473, 1479, 45 U.S.P.Q.2d 1498 (Fed. Cir. 1998) (quoting Lockwood 
v. American Airlines, Inc., 107 F.3d 1565, 1572, 41 U.S.P.Q.2d 1961 (Fed. Cir. 1997)). 

Multi-Format is attempting to take undue advantage of the patent system and its reissue 
procedures to claim entitlement to something it did not invent, in contravention of the written 
description requirement. "The purpose of the written description requirement is to prevent an 
applicant from later asserting that he invented that which he did not[.]" Amgen Inc. v. Hoechst 
Marion Roussel, Inc., 314 F.3d 1313, 1330, 65 U.S.P.Q.2d 1385 (Fed. Cir. 2003). The 
requirement of an "[ajdequate description of the invention guards against the inventor's 
overreaching by insisting that he recount his invention in such detail that his future claims can be 
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determined to be encompassed within his original creation." Vas-Cath, 935 F.2d at 1561 
(quotation and citation omitted). 

Multi-Format is "overreaching" in claiming to have invented an "audiovisual method" 
that includes "performing non-linear editing" on video information before storing the same in 
"compressed" format. When the inventors filed the original application, they only noted in 
passing that "techniques such as non-linear-editing, animation, and special-effects will benefit 
from the implementation of this system," without offering any clue as to how one would actually 
use non-linear editing, e.g., edit before storing in compressed form, or edit the compressed video 
and display the edited version, or store the edited version in an uncompressed form. In short, the 
specification as filed did not contain any description of the presently-claimed process. The. off- 
hand reference in the £ 157 patent to three types of techniques that will purportedly realize a \ 
"benefit" from implementation of the claimed system cannot satisfy the written description 
requirement as to a particularly limited actual use of one of them. 

B. Dependent Reissue Claims 261-263 Suffer Additional Written Description 
Deficiencies 

New reissue claims 261-263 each depend directly from independent claim 256 and 
further limit the recited "audiovisual method" by respectively specifying that the "video and 
audio information" of claim 256 are received by (1) "an interface to a high-bandwidth data 
network" (claim 261), (2) "a satellite receiver" (claim 262) or (3) "a broadcast signal receiver" 
(claim 263). However, there is no disclosure whatsoever in the original ' 1 57 patent that 24 
frames per second video is provided from any of a high-bandwidth data network, a satellite 
receiver or a broadcast signal receiver, and these dependent claims are therefore unpatentable 
under § 1 12, ^ 1 for this additional written description deficiency. 
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In this regard, Multi-Format's reference to column 12, lines 26-42 and Figure 6 of the 
'157 patent (see Response to Protest at 22) does not at all provide support for new claims 261- 
263. The input shown in Figure 6 is film , not a network or receiver . In addition, there is no 
indication that "locally produced materials" are provided from a high-bandwidth data network, 
satellite receiver or broadcast signal receiver. Such items are not local. Column 13, lines 23-26 
of the '157 patent (see Response to Protest at 22) also does not support these claims; there is no 
disclosure that anything received by elements 210-214 is subsequently subjected to "non-linear 
editing" and then stored. Nor is there any disclosure that 24 frames per second video is provided 
by satellite, broadcast television or a high-bandwidth data network. The frame rate of the signals 
supplied to receivers 210, 212 and network interface 214 is unspecified. Indeed, the same 
drawing shows a "graphic processor 242" which converts, inter alia, "frame rates" of the various 
signals received by these units ('157 patent, Col. 13, lines 50-54). Dependent claims 261-263 
are therefore also unpatentable for this additional failure to satisfy the written description 
requirement. 

IV. NEW REISSUE CLAIMS 256-293 ARE UNPATENTABLE IN VIEW OF THE 
PRIOR ART 

It is submitted that all of the new claims 256-293 filed by Multi-Format are unpatentable 
over numerous prior art references, some of which were previously of record. 
A. Identification Of New Prior Art 

The following four prior art references have not previously been made of record during 
the prosecution of the current reissue application (Reissue Application Ser. No. 10/004,046) and 
are attached hereto as Exhibits 1-4. 
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1. U.S. Patent No. 5,355,450 to Garmon et al. 

U.S. Patent No. 5,355,450 to Garmon et al. ("the Garmon patent") issued on October 11, 
1994 from an application filed on April 10, 1992. The Garmon patent is therefore prior art to the 
6 1 57 patent - and thus Multi-Format's pending reissue application - under at least 35 U.S.C. 
§ 102(e). 4 A copy of the Garmon patent is attached hereto as Exhibit 1 . 

The Garmon patent is assigned to Avid Technology, Inc. ("Avid") and is incorporated by 
reference in U.S. Patent No. 5,930,445 to Peters et al. ("the Peters patent") which is also assigned 
to Avid. U.S. Ser. No. 07/866,829, incorporated by reference in the Peters patent (see Peters 
patent, Col. 1, lines 26-30), is the Garmon patent. Due to this incorporation by reference, the 
Garmon disclosure is effectively a part of the Peters patent. See Advanced Display Sys. v. Kent 
State Univ., 212 F.3d 1272, 1282, 54 U.S.P.Q.2d 1673 (Fed. Cir. 2000) ("Material not explicitly . 
contained in the single, prior art document may still be considered for purposes of anticipation if 
that material is incorporated by reference into the document."); Ultradent Prods, v. Life-Like 
Cosmetics, 127 F.3d 1065, 1069, 44 U.S.P.Q.2d 1336 (Fed. Cir. 1997). The Peters patent 
(discussed infra) was one of the primary references cited during the prosecution of Multi- 
Format's original reissue application (now U.S. Patent No. RE 38,079). The first-named 
inventor on the Peters patent (Eric C. Peters) is also one of the named inventors on the Garmon 
patent. 

2. U.S. Patent No. 5,267,351 to Reber et al. 

U.S. Patent No. 5,267,351 to Reber et al. ("the Reber patent") issued on November 30, 
1993 from an application filed on December 22, 1989. The Reber patent is therefore prior art to 

4 Multi-Format's pending reissue application claims priority to the c 157 patent which issued on July 16, 1996 from 
U.S. Patent Appl. Ser. No. 08/298,104 ("the '104 application") which was filed on August 30, 1994. While the M04 
application purports to be a continuation-in-part of U.S. Patent Appl. Ser. No. 08/050,861 ("the '861 application"), 
which was filed on April 21, 1993, there is no claim by Multi-Format that pending reissue claims 256-295 are 
supported by the '861 application. Thus, for purposes of this Protest, it is assumed that the pending reissue claims 
have a priority date no earlier than the August 30. 1994 filing date of the '104 application. 
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the '157 patent - and thus Multi-Format's pending reissue application - under at least §§ 102(a) 
and (e). The Reber patent is also assigned to Avid (and Mr. Peters is also one of the named 
inventors on the Reber patent). This Avid patent contains the following description of non-linear 
editing: 

Non-linear editing on computer oriented systems involves 
digitizing media data recorded from a linear source, e.g., a video 
tape cassette, and storing the digitized media data on a storage 
device, e.g., a hard disk drive. Once digitized, the media data can 
be accessed quickly at any point in the linear sequence in which it 
was recorded so that the various portions of the data can be 
accessed and edited in a non-linear way. 

(Reber patent, Col. 1 , lines 13-21 .) A copy of the Reber patent is attached hereto as Exhibit 2. 

3. Article Entitled "A Real Time, Object Oriented, Non-Linear Editing 
System For Film And Video" by Eric C. Peters 

Eric C. Peters of Avid is a named inventor on each of the above-mentioned Garmon, 
Reber and Peters patents assigned to Avid. In addition, Mr. Peters authored a prior art article 
entitled "A Real Time, Object Oriented, Non-Linear Editing System For Film And Video" ("the 
Peters article") which details non-linear editing and the development of such an editing system ? 
by Avid. The Peters article is cited as prior art on the Peters patent, and with a 
presentation/publication date of October 1989, the Peters article is prior art to the ' 157 patent - 
and thus Multi-Format's pending reissue application - under §§ 102(a) and (b). 

On page 9 of the Peters article, film features, including editing material entirely in 24 
frames per second mode, are discussed. The Peters article also discusses the implementation of 
various effects such as squeeze and pan. A copy of the Peters article is attached hereto as 
Exhibit 3. 
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4. Article Entitled "Pixels and Halide - A Natural Partnership?" by 
David J. Bancroft 

The May 1994 edition of the SMPTE Journal (at pages 306-3 1 1) contains a prior art 
article by David J. Bancroft entitled "Pixels and Halide - A Natural Partnership?" ("the Bancroft 
article"). As set forth on its cover page, the presentation of the Bancroft article was given on 
October 30, 1993 at the SMPTE Technical Conference, and the Bancroft article is therefore prior 
art to the 6 1 57 patent - and thus Multi-Format's pending reissue application - under § 1 02(a). 
While the Bancroft article was cited as prior art during the prosecution of Multi-Format's 
original reissue application (now U.S. Patent No. RE 38,079), it has not been cited to date during 
the prosecution of the reissue application that is the subject of this Protest. 

The Bancroft article describes a system in which film is converted to a common 
production format of 24-frame per second progressive scan video for various processing steps, 
including editing. After processing, the signal is converted to one of numerous different video 
outputs, including 4:3 and 16:9 aspect ratios, anamorphic and HDTV at various frame rates. 
This operation is discussed at page 307 of the Bancroft article and is illustrated in Figure 3 of the 
article. Figure 10 of the Bancroft article also illustrates recording in a 24-frame per second 
format and playing back in several formats. A copy of the Bancroft article is attached hereto as 
Exhibit 4. 

B. References Previously of Record 

The following prior art references are already of record in this reissue application. It is 
submitted that these references, either alone or in combination with one or more of the above- 
listed newly-cited references, render each of the newly-submitted claims 256-293 unpatentable. 
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1. U.S. Patent No. 5,930,445 to Peters et al. 

As discussed above, the Peters patent incorporates by reference the application that 
matured into the Garmon patent, and both are assigned to Avid. The Peters patent is directed to 
an electronic film editing system in which video is digitized at 24 frames per second and edited 
in that format. The Avid system described in the Peters patent is based upon a personal 
computer. Video is compressed and stored in disk storage and then accessed for editing. The 
editing is non-linear because scenes can be selected at random from the disk storage for editing. 
The claims of the Peters patent specifically recite a system for non-linear editing. 

2. U.S. Patent No. 3,617,626 to Bluth et al. 

U.S. Patent No. 3,617,626 to Bluth et al. ("the Bluth patent") discloses a high-definition 
editing and recording system. A high definition camera 1 1 outputs R, B and G components, - 
which are converted to digital by analog-to-digital converter 13 and recorded in digital form by a 
digital recorder 20. The camera is a high-definition electronic camera that operates at 1,575 lines 
per frame, 24 frames per second, non-interlaced. The system described in the Bluth patent 
provides a subsystem that can perform aspect ratio change (e.g., from 2.35:1 to 1.33:1), scene > 
compilation, and scan standards conversion (e.g., for recording to 35 mm color film or 
converting to NTSC or PAL). 

3. U.S. Patent No. 5,335,013 to Faber 

U.S. Patent No. 5,335,013 to Faber ("the Faber patent") also discloses a progressive scan 
video camera operating at 24 frames per second which is converted into a digital format. 
Progressive scanning is employed in the Faber patent, and separate red, green and blue signal 
components are produced. Digital storage of the recorded signal is described. 
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4. Japanese Laid-Open Patent Application No. H04-37846 

According to the translation of the first paragraph of its detailed description, Japanese 
Laid-Open Patent Application No. H04-37846 ("the JP '846 reference") "relates to a method of 
recording and reproducing images, and aims at making the creative operation process . . . related 
to movie production straightforward, accurate and faster ... by means of matching with the 
movie film's standard reproducing speed of 24 frames per second, based on image capture, 
recording and edition [sic] by a video system of 24 frames per second." A copy of the JP '846 
reference and an English translation thereof is attached hereto as Exhibit 5. 

5. The MPEG-1 Standard 

ISO/IEC 11 172 (pts. 1 & 2) (1st ed. Aug. 1, 1993) ("the MPEG-1 standard") discloses 
compression, transmission and storage of digital audio and video signals. 

6. The Van der Meer Article 

A prior art article authored by Jan van der Meer of Philips Consumer Electronics entitled 
"The Full Motion System for CD-I," which appeared in the November 1992 edition of IEEE 
Transactions on Consumer Electronics, Vol. 38, No. 4 ("the Van der Meer article"), describes a 
CD-I (Compact Disc Interactive) system employing MPEG-1 compression. 

7. Japanese Laid-Open Patent Application No. HEI 2-89478 
Japanese Laid-Open Patent Application No. HEI 2-89478 (the "Canon" reference) 

discloses a digital video recorder employing compression. 

C. Applicability of the Prior Art to Multi-Format's Newly-Submitted Claims 
256-293 

1. Claims 256-275 

New reissue application claims 256-275 are directed to a method that includes 
performing non-linear editing on 24 frames per second digital video information and storing the 
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edited information. This is precisely what the prior art system developed by Avid does, as 
reflected in the several Avid references. More particularly, the system described in the Peters 
patent is a non-linear editing system used for editing 24 frames per second digital video. The 
concept of "non-linear editing" is simply that of providing random access to video data so that 
the various portions of the data can be accessed and edited in a sequential way, as described in 
the Reber patent. The system described in the Peters patent certainly performs such non-linear 
editing. 

The Avid system described in the Peters patent stores the results of the editing. 
Specifically, the Garmon patent, which provides part of the Peters disclosure, discloses a 
function called "media consolidate" in which a user can select a set of clips in sequence and copy 
media data to a target disk. The target disk can also be removable. This feature is discussed at 
column 1 1, lines 37-51 of the Garmon patent. Thus, the system of the Peters patent receives 
digital video data having a frame rate of 24 frames per second, performs non-linear editing on 
the data, and stores the edited data. Claim 256 is thus anticipated by the Peters patent (which 
expressly incorporates the Garmon disclosure). 

Even if the Peters and Garmon patents were to be viewed separately - rather than as a 

single reference as they should be given that Peters incorporate Garmon by reference - claim 256 
* 

would be obvious in view of the combination of these two Avid prior art references. Indeed, to 
the extent that the express incorporation by reference of the Garmon disclosure in the Peters 
patent does not form a single disclosure, it unquestionably provides the necessary motivation for 
combining the Peters and Garmon teachings. Moreover, the Reber patent and the Peters article 
are also directed to the same Avid system described in the Peters patent; therefore, it would have 
been readily obvious to combine the teachings of all of these Avid prior art references. 



Appl.No. 10/004,046 



23 



16716-6 



A claim chart illustrating how the Peters patent anticipates claim 256 is shown below 
(with Figure 1 from the Peters patent carrying over to the next page): 







256. An audiovisual method, 
comprising: 




receiving video aid audio 
information, wherein the video 
information embodies a digital 
format having a frame rate of 
substantially 24 fps; 


Computer 18 receives 24 fps video from disc storage 32 ('445, Fig. 1) or 20 ('450, Fig. 1). 

"Disc storage 32 communicates with the computer to provide memory storage or digitized electronic 
image data. This disc storage may be optical, magnetic, or some other suitable media. The editing 
system is user-interfaced via a keyboard 34, or some other suitable user control interface. In operation, 
video and audio source material from a Film which has been transferred to a videotape is received by 
the system via the video tape recorder 20, and is preprocessed and digitized by the audio A/D 22, audio 
processor 24, video A/D 26, and video compressor 30, before being stored in the disc storage 32." 
P445 Col 3 lines 30-4H 

"In operation, the video A/D 26 processes the video signal to reformat the signal so that the video 
represented by the signal corresponds to Film speed, rather than videotape speed. The reformatted 
signal is then digitized, compressed, and stored in the computer for electronic Film editing." ('445, 
Col. 6, lines 57-61) 

"At the completion of this digitization process, the editing system has a complete digital representation 
of the source Film in Film format, i.e., 24 fps[.]" ('445, Col. 7, lines 46-48) 

"Disk storage 20 includes one to seven disks for media storage. The disks may be optical or magnetic. 
The system 10 is controlled by a keyboard 22 and a mechanical user interface 24 to be described in 
more detail herein. In operation, video and audio source material is received by the system 10, 
digitized and stored in the disk storage device 20. The computer 18 is programmed so that the 
digitized source material may be edited and displayed on one of ihe video display devices such as the 
CRT display 12." ('450, Col. 2, lines 18-29) 


performing non-linear editing on 
the video information in its digital 
format to create an edited version 
of the video information; and 


"Once a film is input to the system, a film editor may electronically edit ihe film using the keyboard to 
make edit decision commands." ('445, Col. 3, lines 49-51) 

"A computer-based system for non-linear editing . . .." ('445, Claim 1) 

"As an example of an editing session, one scene could be selected from the bin and played on the 
source window 36 of the system CRT display 12. A film editor could designate Frame points to be 
moved or cut in either timecode or film Footage format. Correspondingly, audio points could be 
designated to be moved or the audio level increased (or decreased)." ('445, Col. 8, lines 38-44; see 
generally '445, Col. 8, lines 20-56) 

See also '450, Abstract. 


storing the edited version of the 
video information on a high- 
capacity storage medium, wherein 
the edited version of the video 
information has a format 
embodying compressed digital 
video having a frame rate of 
substantially 24 fps. 


"Another aspect of the invention is known as media consolidate. Media consolidate allows a user to 
select a set of clips in sequences and then copy media data from the media files referred to by that set 
into new media files on a target disk. A user would typically use this feature when he/she is done or 
almost done with a project and wants to free up most of his disk space but wants to be able to do more 
work at some later date without having to redigitize. By consolidating his media to a single disk, the 
remaining disks can be used for the next project. Of course, if the target disk is removable, all the 
drives in the media composer can be freed up. It is noted that the source media must be on line for 
media consolidate to work since it is not going back to the original tapes." ('450, col. 11, lines 37-48) 
Because the original video used in the editing process was originally in "a format embodying 
compressed digital video," the stored information resulting from the "media consolidate" steps 
likewise will be in compressed form. 
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The claims depending from independent claim 256 (reissue claims 257-274) are similarly 

unpatentable. For example, with respect to claim 257, the Avid system depicted in the Peters 

patent receives non-compressed, non-24 frames per second video data (e.g., 29.97 frames per 

second NTSC video), digitizes it in the video A/D 26, and compresses it by means of the video 

compressor 30 as discussed in the following excerpt: 

In operation, video and audio source material from a film which 
has been transferred to a videotape is received by the system via 
the video tape recorder 20, and is preprocessed and digitized by the 
audio A/D 22, audio processor 24, video A/D 26, and video 
compressor 30, before being stored in the disc storage 32. 

(Peters patent, Col. 3, lines 37-42.) The Peters patent describes this operation as follows: 

In operation, the video A/D 26 processes the video signal to 
reformat the signal so that the video represented by the signal 
corresponds to film speed, rather than videotape speed. The 
reformatted signal is then digitized, compressed, and stored in the 
computer for electronic film editing. 

(Peters patent, Col. 6, lines 57-61 .) The Peters patent then discusses what happens at the end of 
the digitization process: 
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At the completion of this digitization process, the editing system 
has a complete digital representation of the source film in film 
format, i.e., 24 fps[.] 

(Peters patent, Col. 7, lines 46-48.) 

With respect to claim 258, the source of the received information in the Peters patent is 
film having a frame rate of 24 frames per second. With respect to claim 259, the component 
format is a standard format. In addition, the Peters article describing the Avid system 
specifically indicates that RGB or LUV encoding can be employed - both RGB and LUV are 
"component" formats. 

Claim 260 recites that the video and audio information are received from the output of a 
video camera. Although the Peters patent discloses receiving video information from a 
videotape, it certainly would have been obvious to employ its editing system with a similar 24 s 
frames per second video signal from another source such as a camera. 24 frames per second 
cameras were well known by the time of the alleged invention, as shown by each of the Bluth, . 
Faber and JP 4 846 references. Each of these prior art references discloses use of a 24 frame-per- 
second video camera; it would have been obvious to provide the output of such a camera to a 
digital recording and editing system such as that disclosed in the Peters patent, because editing of 
camera outputs was commonplace (the videotapes used in the Peters patent originated with a 
camera). Indeed, both Bluth and the JP '846 reference are specifically directed to systems in 
which editing is also performed. As to the specific recitation of receiving both the video and the 
audio "from the output of a video camera," each of Bluth, Faber and JP '846 will necessarily 
provide audio corresponding to the video, because television and motion pictures provide sound. 
Whether the audio comes from a recorder which is part of the camera or a separate recorder 
synchronized with the camera is irrelevant to the processing performed in the recited method. 
Moreover, essentially every portable video camera has a microphone as well as an image pickup. 
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With respect to claims 261-263, as is the case with a video camera, the source of video 
information to be edited with the system of the Peters patent can be any standard source. It is 
also noted that the MPEG-1 standard discloses the use of 24 frames per second compressed 
video and the use of numerous different transmission mediums. See, e.g., page 1 of Part 2 of the 
MPEG-1 standard. Multi-Format has effectively admitted that transmission of video by network, 
by satellite, and by conventional broadcasting were well known in the art. The '157 patent gives 
no details or instruction as to how these signals are received, or as to how to construct "an 
appropriate adapter unit 220 for the data network or 'information superhighway"' (U.S. Patent 
No. RE 38,079, Col. 13, lines 33-35) or a satellite receiver or broadcast receiver. If these 
transmission modes were not well known, reissue claims 261-263 would lack enablement. By 
presenting claims 261-263, Multi-Format has asserted that these transmission modes were known 
in the art. Thus, all of these transmission modes were art-recognized sources for video, and 
selection of any one known source hardly involves a patentable invention. 

With respect to claim 264, the Peters patent specifically teaches manipulating the video- 
to-NTSC format and outputting the resulting signal for display (Col. 8, lines 44-56). 
Additionally, the Peters article discloses that the Avid system performs various image 
manipulations, such as those described at page 9 of the Peters article. The Bancroft article also 
clearly discloses the manipulation of a stored image into numerous different predetermined 
display formats, as illustrated in Figure 3 of that article. The Bluth patent also discloses 
conversion into numerous different formats, such as 35 mm color film, NTSC and PAL, as 
illustrated in Figure 1 of that prior art reference; and Bluth specifically discloses aspect ratio 
change at column 6, lines 42-61 . It would have been obvious to employ such elements in the 
system of the Peters patent - if that system does not, in fact, already have such elements - since 
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in both cases video data stored in 24 frames per second format needs to be manipulated to 
provide different output display formats. 

With respect to claim 266, the Peters patent uses a 3:2 pulldown conversion to form the 
NTSC output signal displayed (Col. 8, lines 44-56). As to claims 265 and 266, it would have 
been obvious to output an HDTV format and employ 3:2 pulldown in the Peters system. As 
recognized in the specification itself, 30 frames per second HDTV was already a known format 
{see, e.g., Peters patent at Col. 12, line 38 and Col. 10. lines 36-40) and 3:2 pulldown is 
necessary to convert 24 frames per second signals to 30 frames per second signals. Even if it is 
determined that such operations are not anticipated or obvious in view of the Peters patent by 
itself, they certainly are so in view of the Bancroft article, which discloses conversion to 
numerous formats, and the Bluth patent, which teaches that the signal can be converted "for use 
according to one or more of several television standards." (Bluth patent, Col. 9, lines 31-33.) 

With respect to claim 267, the Peters patent employs a general purpose computer. With 
respect to claims 268-275, page 9 of the Peters article describes performing such standard 
operations. Normal image manipulations, such as squeeze and pan, are specifically described in 
the Peters article. 

Independent claim 256 is additionally unpatentable over the Bluth patent, either alone or 
in combination with any of numerous prior art references showing the use of compression (such 
as the MPEG-1 standard or the Peters patent). The Bluth patent discloses a camera 1 1 and A-to- 
D converters that provide a digital input signal recorded by digital recorder 20, a subsystem 
including incremental recorders 50-52, assembler 42 for performing non-linear editing, and a 
master recorder 58 for storing the edited video. The assembler 42 selects various picture frames 
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according to addresses from the incremental recorders to perform non-linear editing and records 
the resulting data on the master recorder 58. 

The correspondence of the Bluth patent to claim 256 is illustrated in detail in the claim 
chart set forth below (with Figure 1 from the Bluth patent carrying over to the next page): 







256. An audiovisual method, 
comprising: 




receiving video and audio 
information, wherein the video 
information embodies a digital 
format having a frame rate of 
substantially 24 fps; 


"This invention relates to an electronic system foi achieving high-definition color motion pictures for 
theatrical and television use, and more particularly to a system of this nature employing digital video 
techniques. " (Col. 1 , lines 5-8) 

"A high-definition electronic camera 1 1 is provided which operates at a scan standard of 
approximately 1575 lines per frame, 24 frames per second, noninterlaced." (Col. 3, lines 31-34) 

"The red, blue and green analog video outputs from the camera control 12 (or from the electronic 
switching system noted above, if used) are converted to digital form by analog-to-digital converters 13, 
one converter being used for each color." (Col. 4, lines 6-10) 

"The output digital signals from the converters 13 are applied to a digital recorder 20 where each 
respective color signal is separately and simultaneously recorded longitudinally on magnetic tape." 
(Col. 4, lines 44-47) 


performing non-linear editing on 
the video information in its digital 
format to create an edited version 
of the video information; and 


"Master tapes which may be copies of the entire tape recorded by the digital recorder, or portions 
thereof, are placed on one or more incremental recorders which in turn allow any number of frames or 
scenes of the recorded motion picture to be selected at will. This is done to enable various frames and 
scenes, as well as special effects, to be assembled into a final master color motion picture." (Col. 2, 
lines 55-62) 

"The recorded information from the digital recorder 20 may be operated upon, modified and assembled 
as briefly noted above by means of a subsystem including three incremental recorders 50 through 52, 
the assembler 42, an assembler control 53, punched-tape reader 54, special effects generator 55, 
member control 56, and a recirculating loop member 57 similar to the memory 24 described above. 
The incremental recorders 50 through 52 are of the type used to record and play back computer data. 
Information concerning the makeup or compilation of the motion picture as decided upon by the editor 
according to his asthetic taste is stored on paper tape which in turn can be played back by the punched- 
tape reader 54. The information from the punched tape allows the assembler 42 to search out the 
various picture frames according to address from the incremental recorders, select the desired 
sequences and record the resulting data on a master recorder 58." (Col. 7, lines 1-17) 

"Although three incremental recorders 50 through 52 are illustrated, any number as desired may be 
employed depending upon how many sequences there may be, or the various combinations and 
permutations of portions of sequences, to be assembled. The assembler then controls the operation of 
each incremental recorder, and selects the various program scenes and frames as needed for the final 
assembled motion picture." (Col. 7, lines 43-49) 


storing the edited version of the 
video information on a high- 
capacity storage medium, wherein 
the edited version of the video 
information has a format 
embodying compressed digital 
video having a frame rate of 
substantially 24 fps. 


"The frame identification data enables playback of desired frames and scenes by the incremental 
recorders 50 through 52 in proper sequence and time as determined by the assembler 42. It will be 
apparent that this then is a sequential assembly subsystem which searches for the particular desired 
scene or frame existing in one of the three incremental recorders aid, in turn, extracts that scene or 
frame and records the same on the master digital recorder 58." (Col. 7, lines 35-42) Thus, the "master 
digital recorder 58 holds the "edited version" of the video. Use of compression is obvious. (See, e.g., 
MPEG-1 standard ("This part of ISO/IEC 1 1 172 was developed in responseto industry needs for an 
efficient way of storing and retrieving audio and video information on digital storage media (DSM) ") 
and the Peters patent) Indeed, the Canon reference uses "image compressing means for compressing 
image data on the basis of characteristics of the image data" (translation, p.6; 12, Fig .1) in the exact 
same environment— a digital video recorder for holdinga high-resolution 24 fps video image. 
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Dependent claims 257-274 are similarly unpatentable over the Bluth patent, either alone 
or in combination with the various references discussed above. The Bluth patent specifically 
discloses that the video information is embodied in a component format, namely RGB, as recited 
in claim 259. Bluth also discloses that the information is received from the output of a video 
camera as set forth in claim 260. In this regard, claims 256 and 260 do not require performance 
of any manipulative step with audio information beyond "receiving" it; these claims thus read 
directly on a system in which the edited and stored video is devoid of audio. Further, although 
the Bluth patent does not specifically discuss audio, this system will naturally provide audio 
recording too since motion pictures and television employ audio; it is a universal practice to edit 
and compress audio along with video. See, e.g., the Peters patent (editing); the MPEG-1 
standard (compression); and the Canon reference (compression). 

Numerous manipulations are performed in the Bluth patent to provide different display 
outputs as recited in claim 264. In addition, the stored video data in the master recorder 58 is 
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high definition and is therefore an HDTV format. Moreover, image processing to provide an 
NTSC output would include performing 3:2 pulldown. As noted above, the Bluth patent 
contains specific discussion regarding resizing the image aspect ratio, which, in the disclosed 
embodiment, is done by cropping. Finally, the Bluth patent also discloses outputting to a film 
recording unit as recited in new claim 275. 

Claim 256 and numerous dependent claims are also anticipated by the Bancroft article. 
This article describes performing editing and processing of 24 frames per second digital video 
signals and subsequent conversion to numerous different outputs. 

The applicability of the Bancroft prior art reference to claim 256 is illustrated in the claim 
chart on the following page (which includes Figure 3 from the Bancroft article): 
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256. An audiovisual method, 
comprising: 



receiving video and audio 
information, wherein the video 
information embodies a digital 
format having a frame rate of 
substantially 24 fps; 



"Figure 3 shows the basic concept - coming out of the telecine is an 'electronic representation* of the 
film. An electronic representaion means several things: Preserving the fim's temporal sampling 
characteristics. A typical film camera can be thought of as capturing all the picture elements of the 
visual image simultaneously in one exposure atthe rate of 24 exposures/sec. To represent this 
faithfully in a video signal we should ideally use a progressive scan 24 frame/sec scheme. Preserving 
the film's spatial characteristics sufficiently for the highest distribution standard needed. To satisfy the 
most demanding video output formats - HDTV formats - an l HDTV order of magnitude of pixels and 
scan lines should be used. An HDTV video distribution derived from film in this way should certainly 
have no less spatial resolution than the HDTV output format allows." (p. 307) Note that use of pixels 
means that the information is digital. 



performing non-linear editing on 
the video information in its digital 
format to create an edited version 
of the video information; and 



"This electronic representation would then travel through color correction, grain and dirt-effect 
reduction, image stabilization and (possibly) editing, just once, to the important-looking box at the end 
- the output format converter." (p. 307) 



storing the edited version of the 
video information on a high- 
capacity storage medium, wherein 
the edited version of the video 
information has a format 
embodying compressed digital 
video having a frame rate of 
substantially 24 fps. 



"This box is the point at which a switch is thrown to decide which video distribution format is to be 
selected, according to which particular market the entertainment product is being sold. What's in this 
box? Well, perhaps not surprisingly, some of the familiar processes that might have been in the 
telecine are in there, such as 3/2 pull-down circuitry, pan and scan or letter-boxing, with de- 
anamorphing of images such as CinemaScope, if desired. The important point is that none of these 
things were locked in, in advance, coming out of the telecine. If the transfer that came out then was 
truly generic to all these various output permutations, then all of them can be derived from the one 
electronic archive copy. This archive copy might be called an 'electronic inter-positive.'" (p. 307) 
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Figure 3. Technology Council proposal: "generic transfer process.* 



The Bancroft article additionally describes many of the limitations set forth in the claims 
dependent from claim 256, including 3:2 pulldown, pan and scan or letterboxing, anamorphing, 
etc. 
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2. Claims 276-284 

New independent claim 276 is directed to a system that includes a video camera for 
capturing digital video information at a frame rate of substantially 24 frames per second and a 
digital video recorder for recording the digital video information at a frame rate of substantially 
24 frames per second. The Bluth patent clearly anticipates claim 276 as it discloses video 
camera 1 1 which captures video at 24 frames per second and digitizes it by A-to-D converter 13, 
and further discloses digital recorder 20 which records the captured digital video. 

The anticipation of claim 276 by the Bluth patent is demonstrated in the claim chart on 
the following page (which includes Figure 1 from the Bluth patent): 
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276. A system for capturing and 
recording digital video information, 
comprising: 



a video camera for capturing digital 
video information at a frame rate of 
substantially 24 frames per second; 
and 



"A high-definition electronic camera 1 1 is provided which operates at a scan standard of 
approximately 1575 lines per frame, 24 frames per second, noninterlaced " (Col. 3, lines 3 1-34) 

"The red, blue and green analog video outputs from the camera control 12 (or from the electronic 
switching system noted above, if used) or converted to digital form by analog-to-digital converters 13, 
one converter being used for each color." (Col. 4, lines 6-10) 



a digital video recorder in 
communication with the graphics 
processor 5 for recording the digital 
video information at a frame rate of 
substantially 24 fps. 



'The output digital signals from the converters 13 are applied to a digital recorder 20 where each 
respective color signal is sqjarately and simultaneously recorded longitudinally on magnetic tape/ 
(Col. 4, lines 44^7) 
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5 The expression "in communication with the graphics processor" in claim 276 has no antecedent basis because no 
"graphics processor" is previously recited in the claim; therefore, reissue claim 276 should be rejected under 35 
U.S.C§ 112,H2. 
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Independent claim 276 is also unpatentable over the JP '846 reference. The first page of 
the translation of this Japanese prior art publication {see Exhibit 5 hereto) states that it performs 
"image capture, recording and addition by a video system of 24 frames per second." On page 2 
of the translation, the "method of video of 24 frames per second" is outlined and the reference 
indicates that it is directed to the "new conception, remodeling of standard generation for video 
cameras." The JP '846 reference also notes the requirement of "remodeling of video-recorder's 
recording/reproducing circuit." 

Thus, the JP '846 reference clearly discloses image capture at 24 frames per second and 
image recording at 24 frames per second. Although the JP '846 reference does not identify 
whether analog or digital capture and recording is employed, it certainly would have been 
obvious to .employ either well-known type of recording with the 24 frames per second method 
disclosed in the JP '846 reference. 

Independent claim 276 is also anticipated by the Faber patent. As discussed above, the 
Faber patent discloses a progressive scan camera 1 10 which operates at 24 frames per second, 
and an analog-to-digital converter 1 14 for converting the output of the camera into a digital 
output. Faber then discloses processing to add grain and modify gray scale and subsequent 
conversion from progressive to interlace in scan converter 128, so as to form a "digital video 
output 130 in a 625 lines/48 fields format" {i.e., 24 frames per second interlaced), which is "input 
directly to a digital video tape recorder 131" (Col. 5, lines 16-27). 

The anticipation of claim 276 by the Faber patent is demonstrated in the claim chart on 
the following page (which includes the corrected Figure 1 of the Faber patent from a Certificate 
of Correction): 
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276. A system for capturing and 
recording digital video information, 
comprising: 



a video camera for capturing digital 
video information at a frame rate of 
substantially 24 frames per second; 
and 



An image 102 is focused by a lens 104 onto an imager 106. The imager is typically a solid state 
charge-coupled device (CCD). However, other imaging devices, such as a pickup tube, may be used 
The synchronization and timing controller 108 controls the scanning, timing and outputting of the 
image signal 1 10 from the imager 106. 

The image signal 1 10 is preferably a progressively-scanned video frame. In comparison, a 
conventional broadcast television camera generates two consecutive hterlaced fields. Furthermore, 24 
frames-per-second of the image signal are generated to simulate the frame rate of a motion picture 
camera. (Col. 4, lines 16-27) 



a digital video recorder in 
communication with the graphics 
processor for recording the digital 
video information at a frame rate of 
substantially 24 fps 



This modified video image is in a 24 progressively-scanned frames-per-second format where each 
frame consists of 625 lines. The image is now converted to a conventional video format. The image 
could also be converted to other video formats as desired. 

For conversion into a 625 lines/48 fields format, the modified video image 126 is inputted into a 
progressive-to-interlace scan converter 128. The scan converter 128 generates two interlaced fields of 
312 and 313 lines each consecutively from the 625 line progressively-scanned frame. The conversion 
is done in the embodiment shown by storing the video image in a memory, and reading out every other 
line for the first field and reading the other lines for the second field. The video image is now 
represented in 48 fields-per-second, each field having 3 12 or 3 13 lines. The digital video output 130 in 
a 625 lines/48 fields format is input directly into a digital video tape recorder 131. (Col. 5, lines 16-28) 
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Dependent claims 277-284 are also unpatentable in view of the prior art references raised 
herein. With respect to claim 277, Faber teaches a graphics processor including a "grain 
simulator" 120, summer 122, and "gray scale modifier" 124, which operate on the digital image. 

As to claims 277 and 281-284, the Bluth patent includes a processor for performing 
image manipulation including resizing the image aspect ratio by cropping. Bluth also employs 
an HDTV format. It would have been obvious to resize by performing a non-linear 
transformation such as squeezing since this was a standard technique as illustrated in the Peters 
article about the Avid system. With respect to claims 278-279, whether the digital recorder is 
employed in the video camera or separate from the video camera is a mere choice from among 
known, art-recognized alternatives. The integrated digital recorder and camera is so 
commonplace that there is an English word for it ("camcorder") and an entire subclass (358/906) 
full of art devoted to it. The use of an optical disk drive for the recorder (claim 280) is likewise 
notorious in the art. For example, it is specifically contemplated in the MPEG-1 standard and the 
Van der Meer article. 

Claims 281-284 are also unpatentable over either the Bluth patent or the Faber patent in 
view of the Bancroft article. Bancroft discloses performing image manipulation as set forth in 
these dependent claims. 

3. Claims 285-293 

Independent method claim 285 is also unpatentable over either the Bluth or Faber patents 
for the same reasons given above with respect to independent apparatus claim 276. 

The anticipation of claim 285 by the Bluth patent is shown in the claim chart on the next 
page (which includes Figure 1 of the Bluth patent): 
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285. A method of capturing and 
storing digital video information, 
comprising: 



capturing digital video information 
at a frame rate of substantially 24 
frames per second; and 



"A high-definition electronic camera 1 1 is provided which operates at a scan standard of 
approximately 1575 lines per frame, 24 frames per second, noninterlaced." (Col. 3, lines 3 1-34) 

"The red, blue and green analog video outputs from the camera control 12 (or from the electronic 
switching system noted above, if used) or converted todigital form by analog-to-digital converters 13, 
one converter being used for each color." (Col. 4, lines 6-10) 



recording the digital video 
information in a format having a 
frame rate of substantially 24 
frames per second. 



"The output digital signals from the converters 13 are applied to a digital recorder 20 where each 
respective color signal is sqparately and simultaneously recorded longitudinally on magnetic tape.* 
(Col. 4, lines 44-47) 
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The anticipation of claim 285 by the Faber patent is shown in the below claim chart 
(which includes the corrected Figure 1 of the Faber patent): 
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285. A method of capturing and 
storing digital video information, 
comprising: 



capturing digital video information 
at a frame rate of substantially 24 
frames per second; and 



An image 102 is focused by a lens 104 onto an imager 106. The imager is typically a solid state 
charge-coupled device (CCD). However, other imaging devices, such as a pickup tube, may be used 
The synchronization and timing controller 108 controls the scanning, timing and outputting of the 
image signal 1 10 from the imager 106. 

The image signal 1 10 is preferably a progressively-scanned video frame. In comparison, a 
conventional broadcast television camera generates two consecutive hterlaced fields. Furthermore, 24 
frames-per-second of the image signal are generated to simulate the frame rate of a motion picture 
camera. (Col. 4, lines 16-27) 



recording the digital video 
information in a format having a 
frame rate of substantially 24 
frames per second. 



This modified video image is in a 24 progressively-scanned frames-per-second format where each 
frame consists of 625 lines. The image is now converted to a conventional video format. The image 
could also be converted to other video formats as desired. 

For conversion into a 625 lines/48 fields format, the modified video image 126 is inputted into a 
progressive-to-interlace scan converter 128. The scan converter 128 generates two interlaced fields of 
312 and 313 lines each consecutively from the 625 line progressively-scanned frame. The conversion 
is done in the embodiment shown by storing the video image in a memory, and reading out every other 
line for the first field and reading the other lines for the second field. The video image is now 
represented in 48 fields-per-second, each field having 312 or 3 13 lines. The digital video output 130 
in a 625 lines/48 fields format is input directly into a digital video tape recorder 13 1 . (Col. 5, lines 16- 
28) 
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Finally, dependent claims 286-293 are similarly unpatentable over the prior art for the 
reasons given above. 
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V. SUMMARY AND CONCLUSION 

In summary, the Protestors submit that all of the claims that are currently pending in 
Reissue Application Ser. No. 10/004,046 are unpatentable for numerous reasons. As detailed 
above, applicant Multi-Format is precluded under the doctrine of restriction acquiescence from 
obtaining claims 256-293, as those claims are directed to subcombinations that are wholly 
distinct from the subcombination that was elected during the prosecution of the original ' 1 57 
patent. Multi-Format is also barred from obtaining claims 256-293 due to recapture estoppel 
because those claims omit a limitation that was relied upon in an effort to obtain allowance of the 
claims of the 6 157 patent and thereby impermissibly seek to recapture surrendered subject matter. 
In addition, claims 256-274 are unpatentable for failing to satisfy the written description 
requirement of 35 U.S.C. § 1 12, ^| 1, while all of the pending claims 256-293 are unpatentable 
over various prior art references, taken singly or in combination. In view of the above, new 
claims 256-293 should be rejected. 

The Patent Office is authorized to charge the cost, if any, of this Protest and/or other fees 
due in connection with the filing of this document to Deposit Account No. 03-1952 referencing 
16716-0000006. 



Dated: 



April 15,2005 



Respectfully submitted: 




David L. Fehrman 



Reg. No. 28,600 
MORRISON & FOERSTER LLP 
555 West Fifth Street, Suite 3500 
Los Angeles, California 90013 
(213) 892-5601 

ATTORNEY FOR PROTESTORS 
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fc _ FIGS. 4<z and 46 are schematic illustrations of the 

MEDIA COMPOSER WITH ADJUSTABLE transition effects screen. 

SOURCE MATERIAL COMPRESSION FIGS. 5a and 5* are schematic illustrations of image 

capture instrumentation. 

BACKGROUND OF THE INVENTION 5 FIG. 6 is an illustration of a keyboard layout 
Video and audio source material editing systems em- . FIG * 7 * a Perspective view of the mechanical user 
ploying digital techniques have been introduced over mterface according to the invention, 
the last several years One example is the Avid/1 Media DESCRIPTION OF THE PREFERRED 

composer from Avid Technology, Inc., of Burlington, EMBODIMENT 
Mass. This media composer receives, digitizes, stores 

and edits video and audio source material. After the reference to F* 0 * 1 ^ media composer system 

source material is digitized and stored, a computer such 10 includes a P 3 ^ of CRT displays 12 and 14 and an 
as an Apple Macintosh based computer manipulates the audio out P ut device 16 which may include two speak- 
stored digital material and a pair of CRT monitors are ^ Thc video d^P 1 ^ 12 311(1 14 and the audio trans- 
used for displaying manipulated material and control 15 dn ^ 16 ^ ^^ed by a computer 18. It is preferred 
information to allow editing to be performed. Later that ^ computer 18 be a Macintosh from Apple Corn- 
versions of the media composer included compression E^ er Corporation designated as II*, II> or Quadra 
techniques to permit the display of full motion video Disk ^^ge apparatus 20 communicates with the 

from the digitized source material. Compression was ^™ p " ter 18 Disk 20 includes one to seven 

achieved using a JPEG chip from C-Cube of Milpitas, 20 m 5 dia stoia « e - ^ disks may be optical or 

Calif. That data compression is described more fully in ma f netlc - The system 10 is controlled by a keyboard 22 
copending application U.S. Ser. No. 07/807,269 filed a mechanical user mterface 24 to be described in 

Dec. 13, 1991, and entitled Buffer and Frame Indexing. m ° re detm \ herem \ „ ^ 
The teachings of this application are incorporated , c . ^deo and audio .source material is re- 

herein by reference. Although previous media compos- 25 T b J ***** stored in me disk 

ers could achieve full motion video from digitized T?S J computer 18 is programmed so 

sources, the compression degraded image quakty below ^ ^digitized source material may be edited and 
desirable levels. Further, the media compose/lacked S^^^fS* .^^eo display deuces such as 
features which enhance the editing procesT lft ^ d * pl f y ^ Tv P lcall y ^ized source material 

6 * 30 would be displayed at a location 26 and edited material 

SUMMARY OF THE INVENTION at a location 28 on the display 12. 

The media composer according to the invention for * /jf r ^ ted the art, 

editing source material includes apparatus for r^W ™ "l 0 * 011 ' 30 ? er 560011(1 color 

digitizlg, storing and editing vktoS „ 5*? ^ 0 ^ ,Wnce " dwnt ?« Ha 20 

materiaL Computing apparatus manipulates theS 35 f^T* * T T ^ ^ 
source material and output apparatus cTmmumc^ and compression of video images. This system includes 
xiAih *h* ~>~,™^„,* _ * /^r. , • \^ a video coprocessor 29, such as the NuVista board 

programmed so that multiple JPEG resolutions can be w *n „,u:^u i . r ~7 e : 

displayed, recorded and pkyed back. W ^ ch °° nverts ^ information uito 

Tr. ^Ttw ♦ <■ !ir * v . digital information, representing each pixel of a frame 

JiS^JSSL J? ^ M° n ' AC withapredetermme^ 

apparatus is programmed to provide motion effects in ^ video coprocessor 29 has a memory 33 which is 

the displayed material and is further programmed to 45 configured using a coprocessor 31, suchi the TI34010. 

provide a *d whose rotation rate corresponds to a niadebTrexas Insets, to provide « outputdau 

sheeted motion effect rate. Motion effects mclude for- patn to feed JPEG circuitry, such as is available as chip 

^ dre rf^ va ™M* ^fed effects, fit-tc-fill capa- CL550B from C-Cube Spitas, Calif. Such configu 

Sif™^ «° U0IL J* UDpr ° Ved r m ? lia T nation can be performed using techniques known in the 

poser of the mvenuon enables a variety of wipes to be 50 art In the system of FIG. 2a, the output data path is 64 

effected, zoom to full screen capability, pitch change bits, divided into four banks of 16 bits. Two significant 

audio scrub, graphics positioning and image capture limitations exist in this situation. First the connection 

u^toumentation. The system also enables sync point path between a video coprocessor 30 and JPEG com. 

editing and slip sync. The system also provides for a pression circuitry 32 was a 50-wire design allowing only 

novel mwmamcal user mterface including a track ball 55 16 bits to pass at one time (16 wires for data; 16 for 

and speed control integrated into a single unit Impor- control of data; others for synchronizing and system 

tantly, the system also supports a media consolidation control). Second, the JPEG drcuitry 32 (and the 16-bit 

process to free Up disk space. connection) was driven by a 50 MHz clock which gov- 

BRIEF DESCRIPTION OF THE DRAWING itS speed to match adequately the 16-bit per cycle 

_ f _ - . v . . 60 flow - The combination of these limitations resulted in an 

llsa schematic diagram of the media composer inability to process 640X480X24 (or 32) bit images at 

sy ?^ m - . 60 fields per second. Typically users of the JPEG chip 

FIG. 2a is a prior art video compression configura- (c3CL550B chip) dealt with this by either shrinking the 

to °crr* ■ - of the image, reducing the bits per pixel informa- 

HG. 2o is the video compression configuration ao 65 tion, or slowing the fields per second rate. AD of this 

cordmg to the present invention. results in lower quality video. 

FIG. 3 is a schematic view of the motion effects FIG. 2b illustrates an improvement over the prior art 

Te ^ n Similar components are used for the video coprocessor 
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29 and JPEG compression circuitry 3Z The memory 33 in a manner known to those of skill in this art. This 

of the video coprocessor 29, however, is configured to dynamic adjustment, or roll-off, is not limited to use 

provide a 12S-bit data path, wherein each pixel is repre- with systems representing pixels with 24-bit words. It 

sented by 24-bits. The connection between the co- may be used with other systems, such as the system of 

processor 29 and the JPEG compression circuitry is run S FIG. 2a. 

at 70 MHz. The JPEG circuitry is programmed using The monitoring of the compression frame size will 

known techniques to indicate that 24-bits of input data is now be described. For this purpose, the coprocessor 31 

used to represent a pixel: The net effect of these im- is programmed, using well-known techniques, so that, 

provements is that the JPEG chip is run faster and at the end of each frame received, eight black lines are 

receives more data, allowing compression of 60 frames 10 provided to the JPEG compression circuit It then is- 

per second of .640x 480x 24 (32) images. sues an interrupt signal, which is received by the host 

The use of a 24-bit word for each pixel may increase computer. By the time the host computer receives the 

compressed frame size depending on the content of a interrupt signal, all data from the compressed frame is 

particular frame. A JPEG chip is configured for com- compressed and the only data remaining in the pipeline 

pression by providing two 8x8 quantization tables, 15 in the JPEG circuit are the extra black lines. Since part 

which are well known in the art The values are placed of the JPEG standard includes placing a marker at the 

into these tables according to frequency. A "Q factor," beginning of the frame, the length of the compressed 

a composite number which multiplies values in the ta- frame may be readily determined. Although the extra 

bles, may be used to designate and scale values in this black lines become part of the compressed image, they 

table. A high Q factor provides increased compression 20 are readily removed upon decompression and playback, 

typically at the expense of resolution. A low Q factor by removing the last eight lines of each decompressed 

improves resolution but typically increases compressed frame. 

frame size. With most systems, if the Q factor is too low, . Table 1 illustrates the various hardware configura- 

and the compressed frame size is too high, the JPEG tions for achieving different levels of resolution. In the 

compression chip cannot compress incoming data at an 25 table, JPEG in refers to the configuration shown in 

adequate speed. Typically, when this happens, the user FIG. 2b. 

is notified, compression stops and a higher Q factor The improved media composer of the invention al- 

must be selected. lows the user to pre- visualize motion effects by creating 

In an embodiment of the present invention, the com- clips and media files which display the requested effect 

pressed frame size is monitored in a manner to be de- 30 The hew clip can be used like any other chip— it can be 

scribed below. When it is determined that the com- trimmed, extracted, overwritten, used in dissolves and 

pressed frame size is too large, the Q factor may be wipes, etc.— and its media can be manipulated as any 

increased, or the tables adjusted, automatically. Con- other, that is, it can be deleted, consolidated, and even 

versely, if the compressed frame size is small, the Q back digitized. The new motion effect clips are video 

factor may be decreased, or the tables changed, to in- 35 only. The start time code is zero hour regardless of the 

crease resolution. The decrease or increase of the Q time code of the original clip. Motion effects can be 

factor may be performed in a binary, step-wise or other created from master clips and subclips, but not from 

suitable manner. Changing the Q factor changes all other motion effect clips. There is a delay as the media 

values in the table, and requires, at playback time, that composer creates new media files. Motion effects are 

the same table be used. As an alternative to changing 40 organized into three related features: variable speed, . 

the Q factor, the values in the table can be decreased for fit-to-fill, and strobe motion. These features are ac- 

the highest frequencies first, and eventually for lower cessed through a single command in a source menu. A 

frequencies, depending on the desired increase or de- Motion Effects command opens a dialog box illustrated 

crease in compression. For this purpose, a table may be in FIG. 3. A preview dial 34 allows the operator to 

maintained to relate a percentage of disparity between 45 pre- visualize the effect even before the OK function 36 

actual and desired compression to a number of values in is clicked on. The dial 34 normally rotates at one revo- 

the table to be decreased. If, for example, , an actual ration per second. When the operator enters values for 

compression of 22 K bytes/frame is achieved when 20 an effect and clicks on PREVIEW, the dial rotates at 

K bytes/frame was desired, a disparity of 10% is ob- the new appropriate speed. In this way, the preview dial 

tained. From this table, the number of frequencies to be 50 works as a metronome to give the operator a feel for the 

decreased can be determined. The change can be made pace or rhythm of the effect 
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TABLE 1 -continued 

Hardware Matrix (PRELIMINARY) 
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Forward and reverse variable speed effects will now 
be discussed. First of all, the operator opens a clip in the 

source monitor 12 and marks an IN and an OUT, and tion effects, the effects are automatically recreated at 
chooses morion effects from the monitor 14 command 20 the end of the process. Motion effects are represented 

menu. The operator then enters any one of three param- correctly in all supported edit decision list (EDL) for- 

eters: duration, play rate (in fps), or percent speed. mats with the following caveat: It may not be possible 

When any one of these parameters is entered, the media to express accurately the media composer motion effect 

composer immediately calculates and displays the val- in an EDL because the format may limit the precision 

ues for the other two parameters. For example, if one 25 with which percent speed can be expressed to whole 

marks a one second piece and enter 50% fps, the media numbers or one decimal place and the media composer 

composer will immediately show two second^ as the is not likewise limited. In this case, the media composer 

duration and 15 fps as the play rate of the new clip. One -, appropriately truncates the rate or percent speed in the 

specifies reverse motion by entering an negative rate or EDL and generates a comment which indicates the true 

percent speed When the operator clicks on OK, the 3 ° rate or percent speed. With the exception of freeze 

media composer creates a new clip and video/media file flames, motion effects are ignored by auto-assembly, 

for the appropriate effect and loads this clip into the Auto-assembly edits video from an original clip at 30 

source monitor 12. It should be noted that when the frames per second, starting at the edit inpoint and filling 

operator asks for slow motion, the media composer the duration of the effect 

creates a clip in which each frame is duplicated a pro- 35 The improved media composer will allow the opera- 

portionate number of times. Similarly, when one asks tor to play four tracks of audio simultaneously instead 

the media composer for fast motion, it creates a cfip in of only two as in earlier versions. The four tracks are 

which some portion of the source frames have been not output through four separate channels, rather only 

deleted When these clips are played, the motion may two. The operator hears the tracks through two speak- 

appear jerky since the media composer is not synthesiz- 40 ers 16 (FIG. 1) according to the pan setting for each 

ing a smoothed series of frames. This effect is especially track. In addition, it is not possible to digitize simulta- 

likely with material transferred from film. neously four channels of audio. The operator can spec- 

The fit-to-fill option allows one to have the media ify which tracks are candidates to be played by tagging 

composer calculate the motion effect required to nil a them with speakers on an edit panel. When the operator* 

specific duration in a sequence with a specific duration 45 chooses 1) to play a sequence, 2) to record a digital cut, 

from the source. The fit-to-fill check box 38 is bold only or 3) to auto assemble a sequence with direct audio, the 

when the operator has marked an IN and OUT in both audio tracks will be output through two channels ac- 

monitors 26 and 28 or the four marks are implied by the cording to the pan setting for every component on each 

location of the position control. Given these values for track. The two output channels can be either analog or 

the source (current) and target (new) durations, the 50 digital according to the wiring of an audio interface 

media composer calculates the necessary rate in percent unit. If the user selects audio from tapes, auto-assembly 

speed of the motion effect One may accept these or will execute edits for audio channels 3 and/or 4. If the 

enter one's own values. Once the media composer has selected EDL format supports four channels, the EDL 

created a new clip, one can overwrite or splice it into will include edits for channels 3 and/or 4. The EDL 

the sequence. 55 formats which support 4 channels are CMX 3600, Sony 

Strobe motion is a type of motion effect in which one 9000 and GVG 4.1. In both the EDL tool and auto- 

frame is held for a specific duration and then the next assembly, the user can specify which media composer 

for the same duration and so on. When one opens a clip tracks are output as channels 1, 2, 3 and 4. One should 

in the source monitor, mark an IN and an OUT, and note that it is not possible to digitize or output four 

select strobe motion 40 in the motion effects dialog box. 60 analog channels of audio simultaneously. However, 

The operator must fill in the n in "update every n regardless of the software limitations, it is not possible 

frames" 4Z The new clip will have the same duration as to output four digital channels of audio because of limi- 

the current one but only even nth frame is displayed A tations in the audio interface. 

Remake Effects command recreates dissolve media files The media composer of the invention will allow the 

and includes all motion effects. As with transition ef- 65 operator to digitize audio at a 48 KHz sample rate, 

fects it is only possible to recreate motion effect media However, it is not possible to use both 48 and 22 KHz 

files when the original source media is online. When the or 44 KHz audio in the same sequence. Thus, 48 KHz 

operator batch digitizes a sequence which includes mo- must be used exclusively when playing a sequence or 
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batch digitizing. When working with the audio inter- 
face and video slave driver and 48 KHz audio is se- 
lected in the digitized selections dialog box, the media 
composer automatically adjusts the sample rate on the 
audio interface. However, one must manually switch 5 
the video slave driver from 44 to 48 KHz. Digitizing 
mixed audio allows one to save space by combining the 
material in two audio channels into a single media file 
which is played from both speakers 16. All other audio 
features, including crossfades, mixdown, and both types 10 
of audio scrub, work with 48 KHz audio. The minimum 
audio hardware required to digitize 48 KHz is a SA-4 
card and either the Pro I/O or Pro Tools. This hard- 
ware is available from Digidesign of Menlo Park, Calif. 
The media composer improves 22 KHz audio by auto- 15 
matically increasing the amplitude of low-level signals. 

The media composer of the invention offers wipes as 
a transition effect Wipes are accessed through a Transi- 
tion Effect command. This command opens a dialog 
box 44 (FIG. 4a) which allows the user to choose in a 20 
pop-up menu between the two transition effects: dis- 
solve and wipe. When wipe is selected, the operator can 
choose a pattern from a menu of sixteen choices 46 
(displayed graphically) and a direction — forward or 
reverse as shown in FIG. 4b. Forward means that the 25 
outgoing clip is represented by the white in the pattern 
from the menu 46 and the incoming by the black (actu- 
ally blue). Reverse means the incoming is represented 
by the white. Regardless of whether the operator 
chooses dissolve or wipe, the duration must be entered 30 
in frames, its start relative to the transition (starting, 
centered, ending, or offset) and a target drive for the 
media files. 

The effect can be viewed only after the media com- 
poser, creates a media file for the specified wipe or dis- 35 
solve. These effect files will be created, deleted, and 
recreated in exactly the same way dissolve media files 
have been in earlier versions. The Remake Effects com- 
mand includes all transition effects. As with motion 
effects, it is only possible to recreate transition effect 40 
media files when the original source media is online. 
For example, media for both the incoming and outgoing 
clip must be online for the media composer to recreate 
the dissolve between them. When a sequence which 
includes transition effects is digitized, the effects are 45 
automatically recreated at the end of the process. All 
wipes are expressed correctly in all EDL formats. A 
dialog box from the EDL Tool allows one to specify 
the appropriate pattern number for each wipe pattern. 
The table of numbers and patterns is stored in a file 30 
which can be moved from on media composer to an- 
other. It is not, however, possible to save and choose 
among several different sets of values. The present 
media composer will also allow the operator to zoom to 
full-screen mode from any monitor (source, record, 55 
pop-up) by pressing the quote key. All keyboard equiv- 
alents function in full-screen mode. The one exception is 
that one cannot use Trim Mode while in full-screen 
play. 

The media composer of the present invention allows 60 
the operator to enter a mode in which a mouse control- 
ler can be used as a shuttle control. Hit L to shuttle 
forward, Play (the 5 and Back Quote Keys) to shuttle at 
30 fps, K (or click a mouse button) to pause (zero 
speed), J to shuttle backward, and the Space Bar to exit 65 
the Shuttle Mode. Hit L twice to shuttle at 60 fps, thrice 
to shuttle at 90 fps. Hit J twice for — 60 and thrice for 
—90. While shuttling at zero speed, either full-screen or 
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normal, many of the keyboard functions are active It is 
possible to step through the program, clear marks, use 
both kinds of audio scrub (see below), go to the next or 
previous edit, show, safe tides, etc If the media com- 
poser cannot do the function and remain in Shuttle 
Mode, the mode is dropped and the function performed. 
Toggling between source and record is an example of 
such an operation. In one embodiment of the present 
invention, jog shuttling may be performed, with a result 
which is similar to that obtained using mechanical jog 
shuttle controls in connection with video tape record- 
ers. With digitized images being played, a mouse or 
similar input device can provide control for jog shut- 
tling. Jog mode begins when an operator pressed a 
button. While the button is depressed, movement of the 
mouse in one direction or another determines the speed 
of shuttling, or of playback. That is, the position of the 
mouse when the button is depressed is used as a refer- 
ence rxjsrrion. With a relationship defined between posi- 
tion and playing speed, the further the operator moves 
the mouse from the reference position, the faster video 
is played back in a given direction. For example, move- 
ment of the mouse to the right increases the forward 
playing speed. If the mouse is returned to the reference 
position, playing stops. As the mouse is moved to the 
left, the reverse playing speed increases. 

An important aspect of the present media composer is 
a pitch change audio scrub feature. When the operator 
shuttles through footage, smooth, continuous audio will 
be produced at the corresponding speed. That is, pitch 
will vary with the speed as with an analog tape. This 
feature is available for one track only. Designate the 
track for smooth scrub by option clicking (or double 
clicking) on the speaker icon for that track. The icon 
becomes an outline. Smooth scrub is available when- 
ever the operator is shuttling, using either the mouse or 
the shuttle control (beneath the Play button on a key- 
board discussed below) to determine the shuttle speed. 

When the operator imports a graphic into the media 
composer and edits it onto the G track of the sequence, 
it may not be positioned optimally with respect to the 
underlying video. When the position control is within 
the graphic element, you can drag the graphic to a more 
desirable position. Option-drag is for fine control and 
control-click will move the graphic to its original cen- 
tered position. During dragging, the media composer 
displays a special window with information about the 
graphic's current position relative to the center (its 
original position) and relative to its position immedi- 
ately before it was moved. Both of these positions are 
measured in pixels along the horizontal and vertical 
axes. 

The improved media composer of the invention has 
been updated to provide image input and output instru- 
mentation in the form of a waveform monitor, a vector- 
scope and black level controls as shown in FIGS. 5a 
and 5b. This improved Video Tool allows the operator 
to save and load settings for contrast, luminance, hue 
and saturation. Such settings control the video co- 
processor 29 and adjust incoming data. The waveform 
generator and vectorscope are analogous to their ana- 
log counterparts which are well known in the art An 
operator viewing the waveform generator and vector- 
scope may use the sliders 100, 101, 102 and 103 to re- 
spectively set values for hue, saturation, luminance and 
contrast These values control the video processor, in a 
manner known in the arts which adjusts incoming data 
accordingly. The line of a frame may be selected for 
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viewing using sKder 104. The use of sliders as an inter- point editing is performed in three steps. First, a point is 

face to allow an operator to set values is well known in specified in the destination, or record, clip, for example 

the art. by placing a mouse cursor on the displayed frame of the 

Keyboard layout is shown in FIG. 6 and the function clip and pressing the mouse button. The location within 

of the keys is set forth in Table 2. The keyboard 22 is 5 the clip is then stored. Second, a point in the source 

augmented by the mechanical user interface 24. The material is identified in a similar manner. Third, the size 

mechanical user interface 24 is shown in more detail in of the source information is specified, for example, by 

FIG. 7. The interface 24 includes a track ball 50 and a using IN and OUT markers. These steps may, in fact, be 

speed controller 52. As with a mouse, the track ball 50 performed in any order, which may be defined by the 

may be used to locate a cursor on the monitors. Buttons 10 programmer of the computer. After these three steps 

54 and 56 serve functions like those provided by the are completed, the source information is overwritten in 

click button on a mouse. It is preferred that the speed the destination, or record, information, by placing the 

control 52 have detents for zero speed normal forward identified source location at the specified destination 

speed (30 frames per second) and a reverse normal location. Such sync point editing may be performed 

speed. It is also preferred that the speed control 52 be 15 with any combination of audio and video clips. Typi- 

spring loaded to return to the zero speed position. It is cally, it is performed to synchronize recorded sound to 

contemplated that additional buttons 58 be provided so an event in a video clip. The operator may then turn 

that a user may program their functionality. PHANTOM marks on in the media composer window 

Yet another aspect of the improved media composer to see how the PHANTOM marks behave in relation to 

is sync point editing which allows the operator to spec- 20 the position control in the record monitor 28. With SPE 

ify a position in one monitor that is desired to be in sync off, the system uses the current position as the IN; with 

with the position in the other monitor. The operator SPE on, the current position is the sync point It should 

then performs an overwrite that preserves that sync be noted that one can mark the IN and OUT in the 

relationship. Sync point editing (SPE) is accessed using record monitor 28 instead of the source monitor 26. 
the SPE command in the media composer menu. Sync 

TABLE 2 

USB Keyboard 



Function 


(y/n) Equivalent 


Notes 


I -Frame Back 




3 


motion control button 


I -Frame Forward 




4 


motion control button 


10- Frame Back 




1 


motion control button 


lO-Frame Forward 




2 


motion control button 


Al track on/off 




9 


track selector 


A2 track on/off 




0 


track selector 


A3 track on/off 






track selector 


A4 track on/off 






track selector 


activate source/ record 
monitor 




ESCAPE 


moved from * on numeric keypad 


All Stop 




SPACEBAR 




dear IN 


Y 


D 




Clear OUT 


Y 


F 




Clear Marks 


Y 


G 




Copy to Cupboard 


Y 


C 




Exit Mouse Shuttle 




SPACE BAR 




Extract 


Y 


X 




Fast Forward 




u 


deck control function 


Find Frame 


Y 


t 




Fall Screen an/off 






mode toggle 


Goto IN 


Y 


Q 


Go to OUT 


Y 


w 




Go to Pre v Edit 


Y 


A 




Go to Next Edit 


Y 


S 




Graphics track on/off 




7 


track selector 


Lift 


Y 


Z 




Mark CHp 


Y 


T 




Mark IN 


Y 


E.I 


two equivalents for convenience 


Mark OUT 


Y 


R,0 


two equivalents for convenience 


Minns 10 Frames 




M 


Trim Mode function-trim buttons 


Minns 1 Frame 




< 


Trim Mode function-trim buttons 


Overwrite 


Y 


B 




Pause 




K 


except in Trim Mode; mouse shuttle and 








deck control function 


Play 


Y 


5 


moved from Tab. The big Play button 








can be configured as Play IN to OUT or 








Shuttle Forward 


Play IN to OUT 


Y 


6 


see Play 


Plus 1 Frame 




> 


Trim Mode function-trim buttons 


Plus 10 Frames 




/ 


Trim Mode function-trim buttons 


Rewind 




Y 


deck control function 


Safe Title/ Action 


Y 


1 




Shuttle Back 




J 


mouse shuttle and deck control function 


Shuttle Forward 




L 


except in Trim Mode; mouse shuttle and 








deck control function 


Sfip Left (1 frame) 


Y 


< 


except in Trim Mode 


Slip Left (10 frames) 




M 


except in Trim Mode 


Slip Right (1 frame) 


Y 


> 


except in Trim Mode 


Sfip Right (10 frames) 




? 


except in Trim Mode 
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Function 



USB Keyboard 
(y/n) Equivalent 



Notes 



Splice 
Trim Both 
Trim Incoming 
Trim Outgoing 
Video track on/off 
Delete Clip/Sequence 
(from a bin) 



V 
L 

k 

8 

DELETE 



only in Trim Mode 
only in Trim Mode 
only in Trim Mode 
track selector 

opens delete dialog box from Bin menu 



Another aspect of the invention is slip-sync editing. 
This kind of editing typically refers to maintaining syn- 
chronization between a series of video clips and corre- 
sponding audio clips when transitions between clips are 
trimmed. In prior art systems, when an audio clip was 15 
trimmed, ie., made shorter, subsequent clips became 
out of synchronization with their corresponding video 
clips. In the present system, when audio material is 
removed from one end of an audio segment from a clip, 
source material from the original audio clip is added to 20 
the other end of the segment so as to maintain the length 
of the audio segment. The source material can readily 
be retrieved from the memory location or disk on which 
it is stored. Thus, the synchronization of subsequent 
clips is maintained. 25 

Another aspect of the invention allows placement of 
graphics material interactively on a frame or frames of 
a video clip. Graphics material may be generated using 
standard, well-known graphics applications programs, 
and may be in standard formats, such as PICT format A 30 
data file for graphics material may be accessed and 
displayed along with a frame from a video clip. Its 
position may be adjusted by placing, for. example, a 
mouse cursor on the graphics. When an appropriate 
position has been determined by an operator, the graph- 35 
ics may be made a permanent part of the video clip. 

Another aspect of the invention is known as media 
consolidate. Media consolidate allows a user to select a 
set of clips in sequences and then copy media data from 
the media files referred to by that set into new media 40 
files on a target disk. A user would typically use this 
feature when he/she is done or almost done with a 
project and wants to free up most of his disk space but 
wants . to be able to do more work at some later date 
without having to redigitize. By consolidating his media 45 
to a single disk, the remaining disks can be used for the 
next project Of course, if the target disk is removable, 
all the drives in the media composer can be freed up. It 
is noted that the source media must be on line for media . 
consolidate to work since it is not going back to the 50 
original tapes. 

What is claimed is: 

1. Media composer for editing source material com- 
prising: 

digitizing apparatus for receiving, and digitizing 55 
video and audio source material; 

storage to receive the video and audio source mate- 
rial; 

computing apparatus including compression appara- 
tus responsive to the digitizing apparatus, said com- 60 
pression apparatus being for compressing and stor- 
ing the source material in the storage, wherein the 
computing apparatus is responsive to the storage to 
determine if the source material occupies more 
than a target amount of the storage and provide an 65 
indication if the source material does occupy more 
than the target amount of the storage, wherein the 
compression apparatus is responsive to the indica- 



tion to adjust its compression if the source material 
does occupy more than the target amount of the 
storage, the computing apparatus further being for 
manipulating the stored source material; . 

output apparatus communicating with the computing 
apparatus for displaying the manipulated source 
material and control information; 

wherein the computing apparatus is programmed so 
that multiple resolutions can be displayed, re- 
corded and played back. 

2. The media composer of claim 1 wherein the com- 
pression apparatus is a JPEG compression apparatus 
and includes dynamic adjustment of tables in a JPEG 
chip, to improve compression roll off. 

3. The media composer of claim 1 further including a 
video coprocessor and including a 128-bit wide data 
path between the JPEG compression apparatus and the 
video coprocessor. 

4. The media composer of claim 1 wherein the com- 
puting apparatus is programmed to provide motion 
effects in the display material and is further pro- 
grammed to provide a dial whose rotation rate corre- 
sponds with a selected motion effect rate. 

5. The media composer of claim 1 wherein the com- 
pression apparatus is a JPEG compression apparatus 
and wherein the clock speed in the JPEG compression 
apparatus runs at 70 MHz. 

6. The media composer of claim 1 wherein the source 
material is digitized to a 24-bit word for each pixel. 

7. The media composer of claim 1 further including a 
mechanical user interface including a track ball and a 
speed controller. 

8. The media composer of claim 1 wherein the com- 
puting apparatus is programmed to generate smooth 
continuous audio having a pitch corresponding to the 
speed of video material being displayed. 

9. The media composer ' of claim 1 wherein the com- 
puting apparatus is further programmed to consolidate 
media to a single disk in the system. 

10. The media composer of claim 1 wherein the dis- 
play apparatus includes a source monitor and a record 
monitor and wherein the computing apparatus is pro- 
grammed so that a video sequence in the source monitor 
may be overwritten onto a sequence in the record moni- 
tor wherein synchronism is maintained between a se- 
lected frame in the source monitor sequence and a se- 
lected frame in the record monitor. 

11. The media composer of claim 1 further including 
a mouse controller connected with a keyboard, the 
mouse controller adapted to function as a shuttle con- 
trol. 

12. The media composer of claim 1 further adapted to 
import a graphic for editing onto a video track wherein 
the position of the graphic may be controlled and its 
position displayed in the display apparatus relative to its 
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original position or to its position immediately before 
the graphic has been moved. 

13. A media composer of claim 1 wherein the com- 
puting apparatus is programmed to display video wave 
forms, a vectorscope and black level controls. 

14. The media composer of claim 1 wherein the com- 
puting apparatus is responsive to the compression appa- 
ratus to receive an interrupt from the compression appa- 
ratus before the compression apparatus has finished 
compressing the source material. 

15. The media composer of claim 14 wherein the 
compression apparatus is adapted to send blank video 
information to the computing apparatus after generat- 
ing an interrupt 

16. The media composer of claim 15 wherein the 
blank video information comprises 8 blank lines. 

17. The media composer of claim 1 wherein the com- 
pression apparatus is a JPEG compression apparatus, 



>5,450 

14 

and wherein the compression apparatus is constructed 
to increase a JPEG queue factor in response to the 
indication. 

18. The media composer of claim 1 wherein the com- 
5 pression apparatus is a JPEG compression apparatus 
and wherein the compression apparatus is instructed to 
adjust compression table values in response to the indi- 
cation. 

l0 19. The media composer of claim 1 wherein the com- 
puting apparatus is constructed to provide a disparity 
value to the compression apparatus. 

20. The media composer of claim 19 wherein the 
compression apparatus is a JPEG compression appara- 

15 tus and wherein the compression processor is respon- 
sive to the disparity value to determine a number of 

frequencies to be decreased in JPEG processing. 
* * * * * 



25 



30 



35 



40 



45 



50 



55 



60 



65 
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ABSTRACT 



A system for the management of media data and index- 
ing of media data based on user instructions. A system 
for the management of relational information between 
media sources is provided as is a method for determin- 
ing media data associated with requests based on source 
identifiers and range specification on the source of the 
data. 

7 Claims, 2 Drawing Sheets 
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The standard or conventional method when estab- 

MEDIA STORAGE AND RETRIEVAL SYSTEM hshing a system of media archival is as follows: As each 

clip of source material is captured for storage in the 

REFERENCE TO MICROFICHE APPENDIX system, the information about the clip and its actual 

The application includes a microfiche appendix pur- 5 *&™A data is either coresident or linked directly at 

suant to 37 CFR §1. 96(b) containing 1 microfiche with ' he ^ of <*P ture ' Whenever the clip Referenced 

76 frames. **y tne user °" t * lc svslcm » " ie media associated with it is 

always the same particular one that was associated with 

BACKGROUND OF THE INVENTION it at the time of the capture (whether the media was 

Tne invention relates to non-Knear editing systems 10 ™ actually was still intact on the original 

and the storage and retrieval of the media associated & ? urce) ' ^ f^TL? C °^T? g ^ 

with the system, i.e.. video and audio data. f h P°J sc «m cnt would directly use the media data tied 

Non-linear editing on computer oriented systems ° U for ^.f playb ? ck ^ formation about 

• „ a1 „^ • ° j- j * j j r v the source that it came from or equivalent sources 

involves digitizing media data recorded from a linear . , . A . . . , ,7 . 

„ * , . . _ ^ . • « « • - 13 would need to be stored with each clip or segment. As 

source, e.g., a video tape cassette, and storing the digi- . . , „ ^ - y * 

^ . , . *\ , . \ . .r. such, the whole collection of clips or segments would 

tized media data on a storage device, e.g., a hard disk . . . fc . „ * . A . . , . 

ru» «u j * \. j be needed at any time in order to determine the breadth 
drive. Once digitized, the media data can be accessed r , a j , 
. , °\. . . A , « . . , . of any source relationships. And as new source relation- 

2,™ fi^LT thC Z ? . Wh,Ch ^! ships were developed it would be difficult if not impos- 

Z ^ Van0US ?° ° f dataCanbe M sible to inform all clips or segments of the new infoma- 

acce^ and edited ma non-linear way. tion. Additionally, tying the media data directly to a 

Editing in either a linear or non-linear system in- „. „_ , „L„w , .„ ._,•„.. 

, . .. . . , _ ... clip or segment would make it necessary to duplicate 

volves a similar pnnaple. Source materia] from some m £ ja i J^ a . emin dj 0f nt$ * verl ^ or 

source (video tape, audio recording, film etc.) is broken were eoaailKi entire)v ^ thin 0 * e 

down into a series of separate clips representing the 2J nt mventkm ^ m(i other Acuities and 

material desired for the final master, and then reassem- problems 

bling these "clips" into a final sequence achieving the 

desire of the editor and producer. "Clips" can be either SUMMARY OF THE INVENTION 
video or audio material or both (synchronous audio and Be invention involves dynamically linking or bind- 
video.) In a non-linear system the typical approach 30 ^ a di ^ tized representation of the media with a spe- 
mvolved aUoting to each clip an associated digitized cific referenC e to the media at the time the information 
section of the ongnal source in storage on the system m b needed at nn ^ m6 being able to ch ^ bind _ 
a media file. The system would allow the user to mg M certain facets m ^ system cnange Xo that end 
manipulate the clips in order to produce the final se- thc mvention is a systcm for Stemming the media 
quence. The clips referred to the media files when cer- 35 needed at the time a clip is request ed to be played, 
tain specific information about the source media was vicwed or information retrieved concerning thc media 
needed, such as the original source name or nature of associated with the clip. Specifically, each clip is dy- 
the media (video or audio), or when the need arose to namically connected to the specific media at the time 
actually view or hear (i.e., play) the media associated that it nee ds access to the media associated with it. 
with the clip. 40 fh c invention also involves the separation of infor- 
For example, a user editing on a non-linear system m ation concerning the specifics of a piece of digitized 
had the ability to manipulate clips into any order, use m edia, information specific about the source material 
audio clips with other video clips, and create new clips the media was derived from, and information concern- 
by using smaller pieces of other clips. Tools existed to ing the connection of media data to those requesting or 
allow the user to combine clips of similar material for 45 needing access to it. Specifically, the three groups of 
other effects. Video clips were used in combination to information that are distinctly separate from each other 
create dissolve effects, and audio clips to create various are: 

audio effects. (1) the information concerning physical source medt- 

Typically, the output of an edit, i.e., an editing proce- urns may indicate which sets (or subsets) of physical 

dure such as the one described above, is an "Edit Deci- 50 source material are equivalent, or make correlations in 

sion List" (EDL) which can be used either by a conven- the labeling of certain segments of the source material 

tional on-line editing system such as the CMX300 or a (example: film edge numbers equivalenced (i.e., corre- 

non-linear system to create or assemble a new linear lated with time code); 

sequence from other existing linear source material, (2) the information about the specific digitized media 

e.g., video tape. The EDL is used to direct the on-line 55 as to the type of media, the length of the data, the range 

system to locate or "cue" the first frame of a desired clip on the source the media represents and the locations of 

which is recorded on a source video tape and loaded such media resources; and 

into a video tape recorder (VTR). The editing system (3) the information concerning the binding of the 
then records the cued clip onto a target or destination media data to the requesters of media. Included in the 
medium, e.g., video tape, and cues the first frame of the 60 invention is the concept that the binding of media re- 
next desired chp. (Note that the next desired clip may be sources to those in need of the media is not made until 
recorded on the same or a different physical source the request for the media is made, and the fulfillment of 
medium as the first clip). Once cued, the editing system the request may change depending on the media avail- 
records the next desired clip onto the target medium. able at the time of the request. 

This process is repeated until the EDL is exhausted and 65 The invention also involves the method of storage 

the target medium represents the selected original mate- and retrieval of the necessary source relational informa- 

rial reorganized into the sequence described by the tion from one invocation of the application to the next, 

EDL. such that it is coresident with the clips and/or media 
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that it is specific for. This makes knowledge of the form possible to sort the records in a variety of orders de- 
of information storage inperceptable to the user of the pending on certain key fields in the records. The list 
system. manager is also able to "sift** certain records from the 

Advantages of such a system are described below: entire pool of records based on requested values of key 

Media need only be digitized once. Clips referring in 5 fields in the records, 
part or in whole to the same media result in references Sectional rile writing utility provides the ability for 
to the same physical data in the system. Duplicate cop- multiple clients to write data to the same file and re- 
ies of the media are not needed or created. trieve it without knowledge of cither the format of the 

Deletion and recapturing of segments of the original jy c or ^ t identity of other clients using the file, 
source results in all clips referring to the specific new 10 

source material entered into the system. MEDIA FILE MANAGER 

Clips requesting media from one physical source may Media File Manager (MFM) is responsible for the 
receive media from a distinctly different physical source management of all media that is present at any time and 
if the sources have been identified as equivalent. available to the system for use. This media may be 

Actual location of the media in storage is free to 15 medifl recorded earlier and stored in some medium, or 
move to any location on disk, without notification nee- available on demand via some link to mechanical de- 
essary to clips requiring reference to the media. Thc media ^ ^ duplicately represented in a 

BRIEF DESCRIPTION OF THE DRAWINGS variety of resolutions or qualities. MFM's purpose is to 
—— m . " . , # . . , n locate media (or information pertaining thereto) sped- 

FIG 1 is a block diagram shown the control and 20 y a request consisting of a range of time from 
media data flow ^ on * m ^ < he some specific source. Tne range in time may be speci- 

source manager, media storage, and media capture and ^ ^ ^ common SMFrE timc ^ fiUn ^ 

HO 8 2Va biock diagram showing the control and numbers real time or ^\ 0 ^^^^^ 
media data flow between the media database and the 25 identification of source material. ** 
table containing media equivalency relationships. s*t of a specific source from which the media is desi eo\ 

° ^ This specific source request is unique and the system 

DESCRIPTION OF PREFERRED EMBODIMENT wor b on the concept that identifiers exist that sepa- 

FIG. 1 is a block diagram illustrating the overall rately identify any source medium, 
functional relationships of the media storage and re- SO At any time after the system is initialized the MFM 
trieval system according to the invention, with media has an internal abbreviation (i.e., a log or set of records) 
hardware, disk storage and the user interface. of all the media that is known to be accessible to it, and 

As shown in FIG. 1, media sources such as VTRs where and how that material may be retrieved. The 
containing linear media data are controlled via editing internal abbreviation is contained in the media database 
and capture systems under the user's control. Digitized 35 built by the MFM. When a client of the MFM requests 
media data from the capture system is transferred and information or access to actual media, MFM uses its 
stored on a mass storage volume such as a hard disk internal abbreviation of the media available to deter- 
drive as digitized media files and selectively retrieved mine a media source that will serve as a satisfactory 
under the control of a media storage and retrieval sys- response to the client's request. A MFM identifier is 
tern which is the subject of the present application. The 40 returned to the client. In any other requests for the 
media storage and retrieval system is implemented pref- specific media the client may use the identifier for quick 
erably as a software application haying two distinct access to the material. This identifier represents the 
components: the media file manager (MFM) and the dynamic link or binding of a client's need for media and 
source manager (SM) along with their respective data- the actual source of media to be used. This identifier 
bases, namely, the media database and the SM database. 45 rcraa j ns constant to the client, and any media deletions, 
A user accesses and operates on the digitized media files changes or additions are changed internally by the 
via calls placed by the editing system to the MFM MFM, i.e., transparently to the user, and have corre- 
which both creates and manages the media files. The spon ding effects on any identifiers already released. As 
MFM also interacts with the SM which inter alia main- described in the procedural interface, two different 
tains a table of relations between the linear media data, 50 t 0 f identifiers can be released by the MFM. For the 
recorded, for example, on source tapes, and the digi- sakc of differentiation, one type is designated a MFM_ 
tized media files. MFM exists in modular software form CRUX m6 Ae ^her a MFM_GIST. the main differ- 
and consists of a procedural interface that accepts re- ^ ^ ^ a lcvc , of privilcgc avai i abl e 

quests for specific pieces of medui from specific source client hoWing 0 ne and/or the other. The holder 

material. SM exists m modular form and consists of a 55 MFM _ CRUX ^ ablc to ^ any number of pro- 
procedural mterface that accepts requests for source information about the 
material relational mformation, and requests to read or c £ ^ actual 
write source relational and specific information to an ^ ^ Usmg a MFM _CRUX, the client 
"^he sourTcode appendix provides specific code for 60 can ^ a routine : mfm open which will pve ^ fct 
implementing both MFM and SM. ThHystem makes a MFM— GIST identifier. Thx ^identifier is a^tedby 
use of two other procedural subsystems: one being a rnfm-read and mfm_elose to be able to read the actual 
linked data list manager, and the other being a sectional media from the file. Tne reason for this is to give the 
file writing and reading manager. These subsystems are MFM some control over which physical connections to 
conventional utilities sometimes ofTered as "toolboxes". 65 actual media are opened or closed. On some systems 

Linked data list management involves the functions this is desired as the number of channels to existing 
of linking together records of information in some or- media may be limited, and MFM needs a method of 
der. Other procedural interactions with this tool make it managing access. 



Exhibit 2, Page 5 



5,267,351 

5 6 

„ Mfm— read is the procedural interface used to pass 

MEDIA FILE PROCEDURAL INTERFACE ^ media ^ fro P m ^ media toto a b Jff cr 

The media file manager (MFM) is a procedural inter- specified by the caller The parameters to the call are 

face between a media requester and the media files designed such that the caller asks for a frame of infor- 

themselves. The interface consists of a set of software 5 mation using a range identifier to identify the frame 

modules which are described below. offset from zero of the range scale. For example in the 

mfm init time code ranging method, the caller asks for the abso- 

Mfm—init is called at the time the application is in- lute time code of the frame desired. The call analyzes 

voked. It is a one-time operation. Its basic functionality the type of media being requested, the size of the buffer 

is to initialize the linked list data structures that will be 10 t he caller has provided and the number of frames the 

needed and determine all media available on the system caller has requested. Based on this information a best fit 

at the current time. Functionally it scans all the disk \$ made for the caller's buffer and the actual number of 

drives on the system and determines if the short-hand frames passed to the buffer is returned, 

version of the media file database which it has previ- mfm— close 

ously placed on the drive is valid. This is determined by 15 Mfm_close is used to allow MFM to close the chan- 

comparing the time stamp on the file with the time ne j t0 fa e media source. The call accepts a MFM 

stamp on the directory in which it was stored. If they q IST identifier and from this identifier MFM is able to 

are equal (they are made equal when the database was distinguish if the media channel is open for write (a 

written) then the database is valid and MFM proceeds p re vious mfm-create call), or open for read (a previous 

to read in the information pertaining to it into RAM 20 mfm— open call). 

using the sectional file routines and then passes the file If ^ media channel j s op en for write, the call exam- 

to the SM (smn readNtable) so that it can read in the ^ a pararocter wmc h indicates caller specific infonna- 

SM information stored there. Of course, the file itself is tiQn abQut the nature of ±Q wrftc to Ae channeL 

not transferred; only its address in memory. If it is uv ^ identifier, media identifier and data rate 

valid then the file is passed to the SM for processing the 25 Qver time are ^ specified . MFM includes this informa- 

SM information contained in it (see mfm quit), and tion m ^ media chaime l m a header and then closes the 

then all media files on the volume are scanned mdividu- ^ m ^ chjmne| (source) ^ now available in , 

ally. In the reading of the media databases and Ae scan- MFM database as a possible candidate for fulfilling 

ning of the other dnves the run tone media database ^ mfm— handle request 

^^m^df 3 ^^ With ^ 0riginal mCdia * 30 8 If on the otherhand the channel was open for read 

Mfm -handle is the call a client, e.g., the user inter- <™ a P^w "^^"^ tffiSS 

face (FIG. 1), uses to receive an identifier (MFM — *> b ™* dosed f " tha * 3,1(1 f »5*J 

CRUX) giving inquiry rights on the media and for de- c ^ nts ™ -»« * c <**™'\ * cn the cl ™± 

media (video or audio,) and the physical channel re- *i f ier wouM be used in further calls to ;mfm_open if the 

quested if any that the source media was recorded from client again desired an open channel to the media data, 
(for instance the type of media may be audio, and the 40 ™ e <*" tnfm-close also makes decisions on the ype 

physical channel may be two, indicating audio2 media). of channel being created. Two types are possible tem- 

To handle the request MFM sifts through its existing IW A temporary media channel 

linked list of records based on the values of the request. exists in memory only for the duration of the apphca ion 

(This is actually done with search support procedures nm. disk resident files are placed on disk and will be 
within the linked list utility.) If a match is found then the 45 available at the time of the next application invocation, 

handle to that record is returned in the form of a For example, an experimental dissolve effect on two 

MFM-CRUX. If no media is found, then MFM calls other video media channels might become a temporary 

SM-relate to determine if any other source material has Ae, while actual video from an externa^ source might be 

equivalent material contained in it, equal to that being captured, digitized and stored in a disk resident file, 
requested by the client. If so, MFM then again sifts its 50 mfm— create 

database, looking for the appropriate media on these Mfm-create is the procedural interface used by the 

other sources. If any is found a MFM-CRUX handle is client who wishes to permanently store media data on 

returned to the client. If no media is obtained via any of «e 30 with the MFM. The call grants the caller a 

these methods, m fm han dle returns a MFM-CRUX MFM— GIST identifier allowing calls to mfra-write to 
for BLACK or SILENCE depending on the media type 55 actually write the media data to a open channel. At the 

originally requested, and flags an error message that no time of the call MFM checks its available space for 

media as requested was available. recording of such information and sets up to receive 

m fm o pen media data into such space. Specifically mfm— create 

Mfm-opcn is called by the client when it will be creates files on disk and preaflocates their size as large 
necessary to actually obtain media data from the media 60 as possible. The initial header on the file is created and 

source. Mfm_open accepts a MFM-CRUX identifier, it is cast a media channel in the stage of creation. In this, 

which the client/requester must have already obtained, way it can be identified later if the caller fails to write 

and proceeds to establish a connection with the media data to the channel, or fails to close the channel (via 

source (open the file). Once the connection is estab- mfm— close.) 
lished the client is given a MFM— GIST identifier. This 65 mfm_ write 

identifier can be used with mfm— read calls to obtain the Mfm— write is the procedural interface used by the 

actual raw media data. caller to actually transfer media data obtained from 

mfm— read source into a media channel of the MFM. It is in this 
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way that MFM is able to store media data for use later 
as a media source in response to mfm handle requests. 

Specifically the call takes in a pointer to a buffer 
containing the data and a length on this particular 
buffer. The data is copied from the buffer for the speci- 
fied length into the media channel identified by the 
MFM_GIST identifier handled in via a previous call to 
mfm_create. The buffer information is simply copied 
onto the end of any data already written to the channel. 



Mfm— quit is the procedural called used by the appli- 
cation when the application is quitting. It provides 
MFM with the opportunity to clean up and write any 
information necessary out to permanent storage before 
quitting. 

Specifically, the list of current media channels known 
to exist is sorted and sifted based on the areas in which 
the channels exist. All the records in the list for those 
channels existing on a single hard disk axe grouped 



The channel may be a temporary channel (main mem- 10 together and written to disk as a single file. This write of 



ory) or a disk resident channel (disk file). The two types 
of records are structured according to the following 
formats. 



Runtime MFM record structure 

One of these is present in memory at ran time for each 

known media file out on disk. 

See C typed ef for MFM_Cmx_t in mfm_pvt.h 



This is the physical channel associated with the media 
type from the physical source. lc: Two tracks of audio from 
the same source would be differentiated by different 
channel identifiers. 
File_ use 

An internal identifier indicating whether the media ilk is 
open for access and if so the nature of the open, read or write. 
Media Type 

This is an internal identifier indication what type of media 
data is stored in the file. Ie. video or audio or some other. 
File Type 

This is an interna) identifier as to the format of the media 
stored m the file. 
Volume ID 
Dtr ID 
Filename 

These three fields together indicate the exact position on 
disk such that the file can be opened, read, written or closed. 
UID 

This is the unique source medium identifier. This is the 
uid that the SM tracks and manages in its database. 
Start—time 

This is the range identifier for the position in the source 
that the beginning of the media data in this file corresponds to. 



End range identifier 
Media— specific 

This is a optional area for storage of specific information 
for this particular type of media. 
Diskfile MFM header 

One of these is present on disk at the beginning of each media file. 
See C typedef for mfm_base_ t in mfm— pvtJi 
Channel identifier 

This is the physical channel associated with the media 
type from the physical source. Ie: Two tracks of audio from 
the same source would be differentiated by different channel 



Media Type 

This is an internal identifier indication what type of n 
data is stored in the file. Ie. video or audio or some other. 
File Type 

This is an internal identifier as to the format of the media 
stored in the file. 
Name 

This is a copy of the character name the user specified 
as the source of the media data m the file. 
UID 

This is the unique source medium identifier. This is the 
uid that the SM tracks and manages in its database. 
Start— time 

This is the range identifier for the position in the source 
that the beginning of the media data in this file corresponds to. 



End range identifier 
Media— specific 

This is a optional area for storage of specific information 
for this particular type of media. 



mfm quit 



information is done using the sectional file write utili- 
ties. Once the database is written to disk in this abbrevi- 
ated form, the file handle is passed to SM (sm_ closeN- 
write) so that SM information pertinent to the media 
15 channels on this disk can also be recorded in the file. 
MFM is unaware of the format or specifics of the SM 
information. Once this is done for all existing disk 
drivec, MFM releases any remaining storage used for its 
run time databases or lists. Control is returned to the 
20 caller (the application). 

Source Manager 

Source Manager(SM) is responsible for the manage- 
ment of all information specific to the physical attri- 

25 butes of the source mediums. Each source medium is 
assigned a numerically unique identifier (UID) upon 
initial exposure to the system. To ensure a unique identi- 
fication, the UID is a random number such as a combi- 
nation of the seconds since Jan. 1, 1904 and the number 

30 of seconds since the system was started. The user speci- 
fied name of the source is coupled with this unique 
identifier. This name/unique id pair is the minimum 
information entered as a record in the Source Manager. 
At its most primitive functional level the SM manages 

35 the list of source medium names being used or referred 
to internally in the system at any point in time.. 

As an additional function the SM is responsible for 
maintaining the relational connections of source medi- 
ums. For example: if two video tapes have been identi- 

40 fied with different names, but actually contain the same 
source video (albeit possibly different time code label- 
ling), the SM is charged with cataloging this informa- 
tion once the user has made the information available. 
In a more specific and restrictive example it may be the 

45 case that only some portion of some source material is 
equivalent to some other portion of some other source 
material. This also the SM is responsible for storing and 
managing. These relationships may exist using different 
standards of labeling. For instance: SM could store the 

50 relationship that a particular piece of film starting at a 
specific edge number is equivalent to some video tape at 
. a time code for some number of frames of video (or feet 
of film). This information is available to clients of the 
. SM for whatever purposes are deemed necessary by the 

55 clients. One such client in the system described here is 
the mfm— handle procedural call in the MFM (See the 
description of mfm handle). 

The run-time SM database is retrieved at each invo- 
cation of the application. Specialized procedures are 

60 used for retrieval and reconstruction of the SM data- 
base. The design and selection of the content of the 
information stored as well as the retrieval method itself 
allow the SM to accommodate changes in the media 
sources available between application runs and mobility 

65 of files describing the edited sequences between systems 
independent of the media associated with them. The SM 
not only keeps track of the location of the media files 
and their sources but also keeps track of varying equiva- 
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lency relationships between the files and portions of SMBuildTable is the procedural interface used by the 

files. Given these properties, the SM functions in such a client to indicate the source identifiers for which the 

way as to never require the user to also be knowledge- pertinent SM information is to be stored in a sectional 

able of the location of this source specific information file manager handle to be indicated later (SMCloseN- 

that the SM maintains. To avoid the need of storing this 5 WriteTable). The client making this call need not be 

information in its own localized place, the SM stores the concerned about indicating the same source identifier 

pertinent pieces of its database in the interested client's multiple times. SM will resolve these duplications later 

disk resident files. At the time of application invocation, when actually writing the file (SMCloseNWriteTab)e). 

as each of these clients accesses or opens the files spe- Specifically, the procedure uses a source identifier 

cific to it, the SM is notified and also reads in the perti- 10 passed in by the client to locate the record in the SM 

nent data to it, stored there at a previous time, or by the database and make an instantiation of the record in the 

SM of another system. fot being built The list being built is indicated by an 

Source Manager Procedural Interface "a*"" P 35 ** ™ s , fct idcntifier was 

derived from a call to SMOpenTable. 

SMInit 15 SMCloseNWriteTable 

Sm_init is the procedural interface the application SMCloseMWriteTable is the procedural interface 
uses upon invocation in order to initialize the SM data ^ by ^ to m( ji C ate to SM that it should use 
structures. It is called once at this time and not again for ^ identifier accumulated in all previous calls 
the duration of the running of the application. t0 SMBuildTable on the specific list identifier, and 

SpedHcally, the call uses the linked list manager to 20 Wlte ^ vaSaaA SM information for those source 
set up data structures to accept records of source names identifiers l0 Ae sectional file manager specified. The 
and source identifiers and for storage of any relational procedur e determines the unique list of identifiers, all 
^formation between these existing sources. The actual information associated with that list and then 

data for the data structures is accumulated at a later ^ mchldes a„ y new names introduced by the process- 
tome. No file is read in from d«k or elsewhere at this 25 . ^ of relationa] jnf ormation . This 
"mS" 'dNT bl S secondary inclusion happens only once and does not 

SMReadNTable is the procedural interface used by. ~ a i^£SL'i' ^S^ZJSX 
^ . . 0 . _ pw . • « . • mation pertinent is then written to the sectional tile 

"522? ° f th ! f ; C " a E r ^ , rl2. m ^K h°- I an manager handle specified, and control returned to the 
added to its data structures. Q.ents of SM who had in a 30 « £ responsibility to close the sec- 
previous application run provided SM the chance to _ * u m 
include information in a file via use of the sectional file ®? 7f s handlt 7 f „ 
utilities use this call to enable the SM to read in the data ^f^'^ PfOCCSS . * a «H th.„ 
and place it in the SM data base. First, the hs of source identifiers sorted and then 
Specifically, the SM processes the call as follows: 35 M * e traversal ensues, all duplications are 
FuTt, the sectional file handle passed in the call is elimmated yieldmg a list of unique records. Also as a 
queried to determine if there is in fact any name infer- of traversal, for each unique source iden ta- 
rnation in the file pertinent to the SM. If there is, the processed, the relational ^formation pertinent to 
length of the data is determined via a call to the sec- that specific source identifier is added to a list of rela- 
tional file manager and then the information is read into 40 information. Once this is completed, the list of 
a block of main memory. Once the data is in main mem- relational information is traversed and processed 
ory the data is traversed and processed. For each record As the list of relational information is traversed, two 
of information it is checked to see if the information is *P"*e" operations take place. First, the relational re- 
already in the SM's database. If it is, it is skipped, if not cord is written to the sectional file manager handle 
it is added. This continues until the entire block of infor- 45 specified in the call. 

mation is exhausted. Once this process is complete, the Second, any new source identifiers encountered are 

file is checked for the presence of any SM relational added to the unique source identifier list. After the 

information. If any exists, a block of memory is alio- relational list is processed, the source identifier list (with 

cated large enough to hold the information and the data source names) is written to the sectional fde manager 

read into it. Once in memory, it is traversed and pro- 50 handle. This completes the process for SMCloseN- 

cessed. In much the same way as previous, any new Write. Control is returned to the caller, 

information is added to the SM's database and redun- SMRelated 

dant or repeated information is ignored. Once the pro- SMRelated is the procedural interface for clients of 

cess is complete, control is returned to the caller, and the SM to obtain information about source relationships 

any memory allocated is returned. 55 (relational information) that SM is managing. This is the 

primary function of the SM to the other systems in the 

SMOpenTable application. For a given source identifier and range 

SMOpenTable is the preliminary procedural call used identification on that source, SMRelated will report any 

by the caller to inform the SM to prepare for building a other source identifiers and ranges that have equivalent 

list of source identifiers. This list of source identifiers 60 media. For example: Let us assume video tape A has a 

will be later used to determine the pertinent information copy (or clip) of some material from tape B on it, and 

to be written to a sectional file handle. that copy of material occurs on tape A from time code 

Specifically, SMOpenTable initializes a linked list to 2 hours 7 minutes to time code 3 hours 27 minutes and 

zero and returns an identifier to the list (ie-, the "list on tape B from time code 4 hours 17 minutes to time 

identifier") to the caller. This identifier is used in subse- 65 code 5 hours 37 minutes. A caller to SMRelated asking 

quent calls to SMBuildTable as the caller encounters for relations to tape B t time code 4 hours 40 minutes for 

source identifiers to be included in the file it is creating. a duration of 20 minutes, (i.e., a portion of the clip) 

SMBuildTable would receive form SMRelated the information that a 
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duplicate exists on tape A, 2 hours 30 minutes for a 
duration of 20 minutes. 

Specifically the procedure works as follows. The 
record in the SM*s database specified by the source 
identifier in the call is located. The relational in forma- 5 
tk>n tagged on that record is sifted for the ranges speci- 
fied in the caller's range specification. A list is con- 
structed of all resultant records using the related source 
identifier and the related range specification. This list is 
a linked list manager list and the handle to it is returned. 10 
Control is returned to the caller. When the application 
quits via MFM—QUIT, the relations created by SMRe- 
lated are written to the MFD 14 on disk by the opera- 
tion SMCloseNWriteTable which is described above. 
In this way, the table of relations is preserved in the 
media database on disk so that its information is easily 
transported with the media database. 

The attached microfiche appendix (incorporated 
herein by reference) embodies the MFM and SM mod- 2 q 
ules. The programming language and compiler used are 
THINK C version 3.01 by Symantec Corporation, and 
the computer used is the Macintosh II running under 
Mac OS version 6.0.2. 

Portions of the disclosure of this patent document and 25 
the accompanying appendix contain material which is 
subject to copyright protection and for which copy- 
right protection is expressly claimed. The copyright 
owner has no objection to the facsimile reproduction, 
e.g., photocopy, by anyone of the patent document as it 30 
appears in the Patent and Trademark Office files, but 
otherwise reserves all copyright rights whatsoever, for 
example, including but not restricted to the right to load 
the software on a computer system. 

In addition to the advantages already discussed 35 
above, the media storage and retrieval system accord- 
ing to the invention accommodates changes and modifi- 
cations to the source of media and makes equivalent 
media available to the client of the method without 
notification to the client. When more complete media 40 
becomes available from another source, or when sec- 
tions of the requested media are absent in the original 
media channel given, the source of the media can be 
changed. 

The system offers the capability of dynamically link- 45 
ing the client of the media with the media available at 
the run time of the application. Such links are possibly 
different with each run of the system, and channels of 
media may actually change even after the link is estab- 
lished. The flexibility of the media management system 
in effect frees the user from making the actual selection 
of source media as well as keeping track of source media 
equivalencies, without unduly burdening the process of 
making the basic identifying request. 55 

FIG. 2 described the digitized media management 
system according to tone aspect of the invention. Data 
is stored in database media files and these files are read 
from the storage device into a working memory where 
a table of relations is built in response to reading the g) 
media files. The working memory accepts a user request 
for media and the requested media file is located in the 
table of relations. A handle is returned to the located 
media file and the media files and the table of relations 
are written from the working memory to the media file 65 
database on a storage device. 

Other embodiments are within the following claims. 

1 claim: 
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1. A method of managing digitized media data stored 
in a plurality of media files in a media file database, 
comprising the steps of: 

reading the media files from the media files database 
located on a storage device into a working mem- 
ory; 

building in the working memory, in response to read- 
ing the media files, a table of relations identifying 
media equivalent to others in at least one common 
subsection by a source identifier that identifies a 
media source and a segment of said media source 
identified by a time range as indicated by lengths, 
frames, time codes or film edge numbers depending 
on the type of indexing used on the source media; 

accepting a request for an operation on a part of a 
specified one of the media files, the part being spec- 
ified in the request by a start time and an end time 
of the specified media file; 

locating the requested media file in the table of rela- 
tions and, if the requested media segment is not 
obtained, locating a media file equivalent to the 
requested media file that satisfies the request; 

returning a handle to the located media file; and 

writing the media files and the table of relations from 
the working memory to the media file database on 
the storage device. 

2. A system for the location and management of 
media subject to user instructions comprising: 

a database in which media form at leas one source is 
stored; 

database means for indexing and referencing the 
media in the database by a source. identifier that 
identifies a media source and a segment of said 
media source identified by a time range as indicated 
by lengths, frames, time codes, or film edge num- 
bers depending on the type of indexing used on the 
source media; 

a table for storing equivalency relationships between 
media; 

table management means, separate from said database 
means, for managing the table of relationships and 
for determining which source identifiers identify 
media equivalent to others in at least one common 
subsection based on time ranges of the source me- 
dia; and 

means for invoking the table management means and 
database means in response to a source identifier 
and time range requested by the user to locate the 
media segment requested by the user and, if the 
requested media segment is not obtained, an equiv- 
alent media segment. 

3. The system of claim 2 wherein said database means 
for indexing and referencing the media in the database is 
dynamic in that it can change or modify the source of 
media to a user without notification to a user. 

4. The system of claim 2 further comprising a means 
for invoking said database means for indexing and refer- 
encing the media in the database to dynamically link a 
user with the media. 

5. The system of claim 2 wherein said means for de- 
termining which source identifiers identify media equiv- 
alent to others in at least one common subsection stores 
only information from its assimilation of data along 
with, but distinctly separate from, data associated with 
the means for indexing and referencing the media in the 
database on request. 

6 . The system of claim 2 wherein said means for de- 
termining which source identifiers identify media equiv- 
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alent to others in at least one common subsection stores 
only information from its assimilation of data along 
with, but distinctly separate from, the storage of infor- 
mation pertaining to segments of source material that 
the user of the system is using to assemble a desired 
edited sequence. 

7. The system of claim 2, further comprising means 10 
for providing for the separation of: 
information pertaining specifically to data about 

source material; 
information pertaining to actual digital or other rep- 
resentations of media available to an editing sys- 
tem; and 

20 



information regarding source segments that the user 
of the editing system is manipulating in creating a 
sequence or producing an edit; 

whereby information pertaining to naming, length, 
positional labeling on the source and associations 
between said labels and labels on other sources 
representing equivalent material and other infor- 
mation pertaining to the source in general, is kept 
separate from other types of information and the 
only connection between a specific source of media 
and the physical source it is derived from is a 
source identifier stored or associated with the 
media representation stored on or available to the 
editing system, so that the only information that a 
user will need to know about such a segment is the 
source identifier it is associated with, the time 
range on said source, and a media type identifier. 
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Pixels and Halide — A Natural Partnership? 



By David J. Bancroft 



A Los Angeles-based body representing the major film studios, the 
Technology Council Oj the Motion Picture and Television Industry, recently 
proposed a change in the way motion-picture productions are transferred 
from film to tape for video distribution. The stimulus for the proposal is that 
the present transfer method is placing an increasing cost and complexity 
burden on the studios because of the proliferating number of video distribu- 
tion formats that have to be derived from each film original; to avoid pic- ' 
ture-quality compromises, each output variant has Typically required afresh 
transfer, implying that the full costs of film handling and labor-intensive 
operations such as color correction have to be borne each time. The 
Technology Council proposes instead that an electronic representation of 
the film original be made that is sufficiently generic to all desired oiaput 
variants that it need be made only once. TJtis article outlines the significant 
technical features of the Technology Council proposal and considers their 
implications and impact from the point of view of a manufacturer of the 
equipment that would be involved, in particular the high-definition telecine 
and videotape recorder. 



Traditionally, film-to-tape transfers 
have followed the video formats of 
the distribution chain: 525/60 for North 
America. 625/50 for Europe, for exam- 
ple. The format selection was always 
made at the telecine and the chosen 
electrical standard persisted right 
through all subsequent processes such 
as color correction, editing etc.. and on 
to final distribution (Fig. I). Another 
selection — which portion of the film 
image to extract for the relatively 
restricted size of the TV image — was 
also made, irrevocably, at the telecine: 
the video output from the telecine 
would therefore have both these selec- 
tions embedded in it. 

The Problem with Film Transfers 
A decision has to be made when it is 
necessary to distribute on more than 
one video standard: assuming that one 
distribution has already been done for 
TV only, it is likely for cost reasons 
thai the editing will have been done in 
video, so there is now an edited video 
master, say 525-line NTSC. but the 
film original remains uncut. So. for the 
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next distribution, to a different market, 
say 625/50 for Europe, the following 
questions apply: 

1. Should the whole transfer (which 
means color correcting, noise reducing, 
image stabilization and editing) be 
done all over again, from the uncut 
negative or imerpositive (IP)? 

2. Should (he negative or IP be 
physically edited before any more 
transfers axe done so that subsequent 
transfers do not require reediting? 

3. Can the corrected, edited and. in 
general, finished 525/60 video master 



be simply standards-converted to 
625/50? 

Options I and 2 do not compromise 
on quality but do incur the handling 
and color correction costs of the trans- 
fer all over again. Option 3 is cheaper, 
but audiences in Europe complained 
about the quality when this was tried 
with "Dallas" some years ago (3/2 
pulldowns at 59.94 Hz interpolated 
down to 50 Hz do not iook pleasant!). 
None of these options is particularly 
attractive. Even if the transfer is of a 
completed feature film, as opposed to 
footage destined for editing into. say. a 
TV series, the labor and expense of 
redoing the color correction each lime 
is formidable ("color correcting" is 
really far more than the term implies 
' — a large pan of it is deciding which 
pan of the large print density range of 
the film to capture on the smaller 
dynamic range of the TV system, often 
on a scene-by-scene basis). 

" Now ihe problem is becoming 
worse: today, it is not just a question of 
two or three distribution formats — 
there may be dozens, if we consider all 
the permutations. It is a question not 
only of video scanning formats, bui of 
optical formats too (e.g., pan and scan 
versus letterbox, optical letterbox with- 
in a 4:3 raster. 16:9 anamorphically 
contained within a 4:3 raster. 16:9 




Figure 1. Today's situation: one full pass per output format. 
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inside a raster designed for lo.,, etc), 
and now HDTV is joining the fray. 
Figure 2 shows just a few of the myri- 
ad combinations wiih which a transfer 
facility mignt have to deal. 

Something obviously has to be done 
about this, and this article gives the 
viewpoint of a manufacturer respond- . 
ing to a solution proposed by the 
Technology Council of the Motion 
Picture and Television industry at the 
1993 Montreux Symposium. 1 This was 
further reported at the 1 35th SMPTE 
Conference by one of the signatories to 
this proposal. 2 

The Technology Council 
Proposal 

. The Technology Council in effect 
proposes that instead of selecting the 
output standard for the transfer right at 
the telecine. as shown in Fig. 1. the 
decision should be deferred until the 
last moment. What does that mean in 
practice? 

Figure 3 shows the basic concept — 
coming out of the telecine is an "elec- 
tronic representation" of the film. An 
electronic representation means several 
things: 

• Preserving the film's temporal 
sampling characteristics. A typical 
film camera can be thought of as cap* 
luring all the picture elements of the 
visual image simultaneously in one 
exposure at the rate of 24 exposures/ 
sec. To represent this faithfully in a 
video signal we should ideally use a 
progressive scan 24 frame/sec scheme. 

• Preserving the film's spatial char- 
acteristics sufficiently for the highest 
distribution standard needed To satis- 
fy the most demanding video output 
formats — HDTV formats — an 
"HDTV" order of magnitude of pixels 
and scan lines should be used. An 
HDTV video distribution derived from 
film in this way should certainly have 
no less spatial resolution than the 
HDTV output format allows. 

• Preserving the film 's optical for- 
mat (exposed or printed area) suffi- 
ciently for all output variants. 
Considered at the most simple level, 
that would mean scanning perforation* 
;o-perforaiion J horizontally and frame - 
:op to frame-bottom vertically. This 
*ould be the only sure way to accom- 
modate without cropping the vast array 
>f film formats residing in film 
ibraries today. 4 

This electronic representation 
*ould then travel through color cor- 
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Figure 3. Technology Council proposal: 'generic transfer process.' 



rection. grain and dirt-efTect reduction, 
image stabilization and (possibly) edit- 
ing, just once, to the important-look- 
ing box at die end — the output format 
converter. 

This box is the point at which a 
switch is thrown to decide which video 
distribution format is to be selected, 
according to which particular market 
the entertainment product is being 
sold. What's in this box? Well, perhaps 
not surprisingly, some of the familiar 
processes that might have been in the 
telecine are in there, such as 3/2. pull- 
down circuitry, pan and scan or letter- 
boxing, with de-anamorphing of 
images such as CinemaScope. if 
desired. The important point is that 
none of these things were locked in. in 
advance, coming out of the telecine. If 
the transfer that came out then was 
truly generic to all these various output 
permutations, then all of them can be 
derived from the one electronic archive 



copy. This archive copy might be 
called an "electronic inter-positive." 

Problems in Implementing the 
Technology Council Proposal 

Now, that's the ideal — how close 
could we come to this in practice? 
WelK there are some limitations, of 
course. Let* s look at some of die film 
formats to see if we can find the 
extreme cases. 

Figure 4 (upper portion) depicts the 
first challenge. Here the problem is in 
the vertical axis. The film was shot for 
'1.85:1 Academy distribution, but pro- 
tected for 1.33:1 (4:3) television as 
-well. What is meant here is that 
although the action would have been 
framed for the 1.85:1 slice in the mid- 
dle, the director would also have 
ensured that no alien objects such as 
lights or mike booms intruded, into the 
areas above and below. Then, when the 
x film gets transferred for conventional 
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Figure 5. Single trvry generic transfer format. 

4:3 television, instead of clipping off 
the sides, these areas above and below 
are revealed. So now we have the prob- 
lem that because all of the vertical 
height has to be scanned to support this 
4:3 output, only a fraction of the total . 
scan lines is available lor the 1.78:1 
(16:9) image. That fraction is about 3/4. 
In the 16:9 case, which is needed lor 
HDTV distribution, the image really 
must have full vertical resolution — 
possibly over 1000 lines. So we have to 
multiply the number of wanted lines by 
the inverse of this fraction, i.e.. by 1.33. 
to get the number of lines pumped into 
the generic electronic master up high 
enough to yield the lull measure in this 
truncated 1.78:1 output. Thai mnkes 
about 1400 to 1500 lines thai need to i>c 
captured from the film.* 

Figure 4 (lower portion > shows the 
.other challenge — CincmaScope. On 
the film, the image looks almost 
square, but it really represents a 2.35:1 
image, because it's been shot with an 
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anamorphic lens on the camera that 
squeezes the width of the scene into 
the available film width and anticipates 
a corresponding lens on the projector 
to expand it back out again. This 
expansion is needed in a video distrib- 
ution chain as well — either in the 
telecine or, in this new proposal, in the 
output format converter box, as previ- 
ously mentioned. Unlike the cinema 
screen, however, even the new 
widescreen video formats can only 
make a 16:9 image, or 1.78:1 — they 
can t make 2.35:1. so that means either 
a letterbox must be created, or — the 
real challenge — a pan and scan image 
to make 1 .78: 1 out of 2.35: 1 . To main- 
tain full resolution in the pan and scan 
option it is necessary to store 
2.35/1.78. or about 1.32 times the 
desired final output number of 1920 
samples/line in our electronic archive 
to achieve this — somewhere around 
2500 active samples/line. 6 
These two cases probably represent 



the two extremes of all the film for- 
mats likely to be encountered in prac- 
tice. Between them they might seem to 
be dictating that the Him should be 
scanned as in Fig. 5. in a format of 
about 2500 pixels X 1500 lines. 
However, fortuitously, these two 
extremes never do in fact occur at the 
same time — when we need to over- 
sample horizontally with 2538 it is not 
necessary to oversample vertically 
with 1440 or 1536. This is shown in 
Fig. 6. which illustrates a compromise. 
What is fortuitous is that each of these 
two cases creates about the same 
amount of data for the VTR and other 
downstream equipment to handle. 
What is less fortuitous is that there are 
now two formats instead of one. But 
that does not mean we have lost our 
goal of only one transfer per film, 
because each film only needs one 
transfer of the appropriate type, it does 
mean, however, that the telecine and 
all following equipment, such as the 
VTR. has to be able to accept two for- 
• mats instead of one. 

What It Means to Manufacturers 

How should manufacturers react to 
this? First, even with the compromise 
dual-format scenario, we are looking at 
an obviously bigger image sampling 
array than in any of the HDTV Formats 
that the world has seen so far; the 
implications for image sensors, and for 
equipment that has to store and manip- 
ulate a now-greater-number of pixels 
are quite profound. Second, just the 
fact that this sampling amay. together 
with other attributes such as frame rate 
and progressive versus interlaced scan, 
are different — from what has so far 
been etched in silicon and embedded 
into signal processing architectures and 
tape formats — is significant. It is not 
a trivial matter to develop revised ver- 
sions of products containing such hard- 
ware. 

One mitigating factor is that the 
requested frame rate, 24 frames/sec. is 
20% lower than 30 frames/sec. That 
does not help much with the image 
sensor, but it does with everything 
else. It could be said that every piece 
of digital processing or storage equip- 
ment has a fixed data rate budget — 
spend less on the temporal domain and 
there is more left over for the spatial 
domain: this would be matching the 
characteristics of the film source mate- 
rial anyway. Would the increase in 
spatial data caused by the use of 2538 
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X K)80 or 1920 X 1440 sampling 
arrays stay within ihis 20% extra 
allowance? Comparing such arrays to. 
say. 1920 X 1035. an established 
HDTV format, that would be about 
2.75 million active samples/frame ver- 
sus just under 2 million. Unfortunately, 
lhai means not a 20% increase but a 
37% increase, going beyond the appar- 
ent capabilities of today's HDTV 
equipment. 

Options 

What are the options at this point? 
One suggestion is compression. The 
argument goes that the Him communi- 
ty has accepted the latest technological 
evolution from the video community: 
data compression in small amounts: a 
ratio of 2: 1 already seems to be accept- 
able. If 2:1 compression is acceptable 
in an interlaced scan formal. 4:1 may 
look just as good in progressive scan. 
Another suggestion, perhaps less radi- 
cal, is that 2: 1 compression of the color 
difference signals is already utilized in 
the ITU/CCIR 601' standard-definition 
digital format (4:2:2 sampling), 
because of the subsampiing compared 
to luminance, but that factor is limited 
to the horizontal axis because of inter- 
lace. With progressive scan, the verti- 
cal axis can be subsampled as well 
(4:2:0 structure), saving 25^ of the 
total data compared to a 601-like sam- 
pling structure. Putting this additional 
subsampiing together with the reduced 
frame rate would give a total saving ol 
40% — enough to accommodate the 
extra spatial samples. 

Relationship to Other Parts or 
the Production Chain 

Another issue is the question of the 
interface between this format, wiih its 
strong 24-frame characteristic, and the 
formats of the various output distribu- 
tion chains. Of particular consideration 
are the likely new formats for electron- 
ic production of HDTV in the U.S. 
One of the strong contenders, which 
has already been noted by the 
Advanced Television Svsiems 
Committee <ATSQ« as an intcnm for- 
mat for production, is based on a sam- 
pling array of 1920 X 1080. with a 
temporal rate of 60 interlaced 
fields/sec. This is obviously designed 
for an easy interface to U.S. distribu- 
tion standards, both now and in the 
future. How can a 24-frame format be 
related to that? 

SMPTE Journal, May 1994 
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One answer is that telecines a!! 
across North America. Japan, and 
other MTSC countries have been bridg- 
ing the 24 frames-io-60 fields gap for 
decades, with the familiar 3/2 pull- 
down technique. The only difference 
here is that the bridge would be occur- 
ring later on in the production chain. 
However, there is also the more subtle 
question of the image sampling array. 

One argument could be thai the f,| m 
transfer formal should use the same 
sampling array, or common image for. 
mat. as the ATSC interim format of 
1920 X 1080 previously mentioned. 



The problem with this is that although 
it might simplify manufacturing, 
because of the reduced rather than 
increased daia rate, it would not handle 
CinemaScope and 1:85. film with the 
resolution levels defined earlier. 

This leads to an alternative concept 
of a common data rate format, in 
which the pixel count is increased to 
take advantage of the lower frame 
rare, then increased further to achieve 
Technology Council capabilities, but 
with the increase being compensated 
with some modest compression until a 
clock frequency that matches today's 
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Figure 1 1. Details ot output standard convener — US. HDTV case. 



proposed HD production formats is 
achieved- This would simplify the 
design of switchable equipment, as 
well as going some way towards 
meeting those original Technology 
Council criteria. This idea is illustrat- 
ed in Fig. 7. 
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Illustrating the Concept — 
Regular-Definition TV 

As a way to illustrate the concept 
more simply, consider first the use of 
625/50 as the core recording formal. 
As shown in Fig. 8. the telecine plays 
the 24-frame film at 25 frames (4% 



faster), onto a 625/50 VTR. Of course, 
that's interlaced, so the telecine is hav- 
ing to perform what might be called a 
2/2 pulldown. That matches European 
or other 625 distribution as it stands. 
For the U.S. and other 525-iinc coun- 
tries.. a little more work is required to 
get to 30 frames 9 — it is necessary to 
get back to 24 frames and then use the 
established 3/2 pulldown technique. 
However, this is not a telecine — this 
is a 625/50 VTR. What is needed is to 
play it at 24 frames/sec or. more cor- 
rectly, 48 fields/sec. by slowing down 
all its clock references. Thai also hap- 
pens to cancel out the 4% speed error 
incurred in the original transfer, but 
most importantly provides the input 
needed to supply -to a fairly simple 
converter that just repeats fields in a 
3/2 sequence, just as the telecine would 
have done. 

Having corrected the frame rate, the 
remaining task of making 525 tines out 
of a 625 raster is a relatively simple 
and clean process. Both major televi- 
sion markets therefore receive a video 
copy of the film without either one suf- 
fering from or paying for any field 
* interpolation. Thus it might be consid- 
ered to be a traditional 525/625 
telecine, but with a tape buffer inside, 
so the film can be removed and put 
safely away after the first transfer. 
Then, the format-setting switches on 
the tape buffer arc simply set to play 
out transfers in the various TV stan-. . 
dards desired. The "buffer" just has to 
be good enough for whichever is the 
most demanding standard. 

Extending Capability to HDTV 

That is the principle illustrated using 
standard-definition TV distribution. 
Figure 9 shows a scheme that extends 
this to HDTV by using not a 625/50 
VTR but a 1250/50 VTR. Consider the 
outputs going to the U.S. Again, by 
playing back the tape at approximately 
48 fields/sec instead of 50 fields/sec. 
the basis is established for a 3/2 pull- 
down sequence to make 30 frames or 
60 fields/sec. There are some motion 
artifacts, but they are no different than 
those created by a telecine making a 
30- frame video output via the same 
pulldown technique. For the raster con- 
version, to make 1080, 1035, 960. or 
whatever, out of the 1 152 active lines 
of the 1250 format is not difficult or 
lossy — converters already exist that 
do this cleanly. A 525-line raster is a 
subset, so is again very easy. For 
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Europe, a 1250/50 copy already exists; 
finally, for the 625/50 standard, a 
1250-IO-625 downconverter is a com- 
paratively simple and lossless device. 

What are the disadvantages? First, 
the signal is interlaced, not progres- 
sively scanned. This is not a problem 
for prcseni-day TV distribution, but for 
future systems where progressive scan 
frames may travel all the way to the 
home, the fields would have to be 
assembled back into frames again by a 
converter before transmission. 
However, because each pair of fields 
originates from the same film frame 
and therefore represents the same 
instant in time, no additional artifacts 
would be produced by such dei menac- 
ing. 

The primary advantage of using an 
existing formal such as 1250/50 is that 
it exists now; telecine machines 10 and 
digital 1250/50 VTRs" are now 
appearing on the market, for example. 
A secondary advantage is thai the full 
gamut of standard-definition 625/50 
equipment is available for all offline 
decision- making processes at very rea- 
sonable prices, since the development 
costs have already been amortized in a 
large existing marketplace. 

Being TYuly Technology Council 
Compliant 

For the 24- frame idea, however. Fig. 
10 illustrates the same principle as in 
the 625-line and 1250-line schemes. 
The disadvantage is that no TV output 
can be obtained without some conver- 
sion (in Figs. 8 and 9 the 50 Held/sec 
outputs were available immediately). 
One advantage, however, is that a true 
24-frame representation can serve 
more than just television distribution: 
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'electronic cinema, for example, might 
work very well with a 24 frame/sec 
progressive scan signal projected with 
the electronic equivalent of a multi- 
blade shutter — possibly a 72-Hz 
refresh to abolish flicker on a large- 
screen display where the brightness 
might be much higher than on today's 
traditional cinema screens. 

Figure 1 1 illustrates in detail what 
would go into the ourput standard con- 
verter referred to earlier. The order of 
processing shown here is intended only 
to demonstrate the concepts; a manufac- 
turer of the actual device might plan the 
architecture differently, but these 
processes would be in there somewhere. 

This leads to Fig. 12. which summa- 
rizes the functions that would be relo- 
cated from the telecine to the output 
converter as a result of adoption of the 
Technology Council scheme. x 

Finally, Fig. 13 presents the chal- 
lenge to manufacturers — mostly in 
the form of more pixels, more data, 
and revisions to existing architectures 
that have been derived from broadcast 
standards. 
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Conclusion 

It may be that a -compromise will 
have to be reached between these 
Technology Council ideals and what 
manufacturers can actually build, but 
ultimately, no manufacturer can push 
his format, or any particular format, 
upon the production or post- production 
community. The users must decide on 
the most harmonious partnership 
between pixels and halide. 
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(1 l)Publication number : 04-037846 
(43)Date of .publication of application : 07.02.1992 




(51)Int.CI. 



G03C 5/12 



. G03B 



27/48 
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5/253 




(2 1) Application number : 02-144373 (71)Applicant : HIYAMA SHIGEO 

(22) Date of filing ; 04.06.1990 (72)Inventor : HIYAMA SHIGEO 



(54) PRODUCTION OF MOVIE BY 24 FRAMES/SEC VIDEO SYSTEM 

(57)Abstract: 

PURPOSE: To easily, surely and rapidly execute the working process of movie 
production by matching photographing, picture recording and editing based upon the 
video system with the reference reproducing speed of a movie film. 
CONSTITUTION: When a video camera, a video recorder and a video display are 
modified and the synchronism and phase coupling of these-modified units with a film 
projector is set up, film images are converted into video images at the speed of 24 
frames/sec after telecine processing the film and the film can be held at the same state s 
film reproduction. Thus, problems such as a real time difference between a movie film 
and video images and the movement, flow and flickers of pictures can be resolved. 
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JAPANESE LAID-OPEN PATENT APPLICATION HG4-37846 
FILED ON JUNE 4, 1990 
PUBLISHED ON FEBRUARY 7, 1992 

5 SPECIFICATION 
.1. Title of the Invention 

MOVIE PRODUCTION METHOD BY A VIDEO SYSTEM OF 24 FRAMES PER 
SECOND 
2. Scope of claims: 
10 (1) . A movie production method of carrying out conventional 
positive editing, negative editing and the like based on video 
edition of film images captured for movie production 
telecine-processed at 24 frames per second. 

15 3. Detailed Description of the Invention 

The present invention relates to a method of recording 
and reproducing images, and aims at making the creative 
operation process (mainly edition and dubbing) related to 
movie production straightforward, accurate and faster, as 

20 well as protecting copyrights ant the like, by means of 
matching with the movie. film's standard reproducing speed of 
24 frames per second, based on image capture, recording and 
• edition by a video system of 24 frames per second. 

Recording/reproducing of colored conventional video is 

25 carried out at 29.97 frames/s (in general, it is said to be 
30 frames/s) , and although the telecine has been remodeled 
to the extent of being visually acceptable at the projector 
side as it can - be mainly observed in the 2-3 pull-down 
methodology, there are the following fundamental problems 

30 left unsolved. 

□ The problem related with substantial time difference in 
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making correspondence of respective single pictures 
between a movie film reproduced at 24 fps and video images 
of 30 fps. 

□ The problem of movement, slip and flicker of the actual 
5 picture.. 

From the above situation, there were fundamental 
problems left in using video within the movie production 
process. 

The present invention aims . at performing movie 
10 production without the above-mentioned drawbacks ® and (D, 
and solving them through the method described below. 

In other words, it solves the problems by setting the 
standard projection speed to 24 fps by utilizing a method of 
a 24 fps video when telecine-processing a film. 
!5 Method of video of 24 frames per second 

O New conception, remodeling of standard generation for 
video cameras. 

O Remodeling of video-recorder's recording/reproducing 
circuit. 

20 O Remodeling of video display (CRT, video projector) 

O Synchronization with film projector, phase-coupling 
setting. 

From the above, the film image is converted to video at 24 
fps after telecine-processing, thus permitting- preserving 
25 the same conditions as the film reproduction. 

As a result, the aforementioned problems (D and (D are 
solved, leading to benefits for movie production as follows. 
First, 

O In the. image-shooting site, by removing video images from 
30 the film camera through this process, it is possible to 

perform recording/simultaneous reproducing of the 
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original pictures at 24 fps without flicker. In addition, 
by conveniently setting the image formation phase between 
the film camera and the video camera, it becomes useful' 
in recording/reproducing/verifying ''special effects" or 
5 . the like, which could not be regularly captured. 

O As for editing, by applying. the video editing according 
to the present system to the conventional positive editing 
for movie production, the process becomes straightforward, 
fast and absolutely free of errors- between the film frame 

10 and video frame, thus presenting an accuracy that makes 

the process resilient against movie editing of hundreds 
of cuts and its processing. The process that goes from 
the so-called preliminary edition through rough cut to 
all-rush edition will all correspond through the video 

15 according to the present system. In addition, 

representation is made possible including full-fledged 
video processing, thus permitting to concentrate ideas on 
the filming process. This will especially allow the staff 
to perform presentations of various versions. These facts 

20 will give room to participation of images from the 

production staff into each operation production mode of 
the Japanese movie system of one-camera style, thus 
resulting, in improving the communication between staff, 
related people, advisers, etc. Performing video editing, 

25 processing, special effects and the like at this process 

makes the present system further effective for the reasons 
mentioned below. By utilizing the time code address, 
which is bound to become general rule in the current 30 
fps video world, coupled with the film time code and the 

30 video system of 24 fps, which is still not actively utilized, 

it becomes possible to carry out repetitive operation 
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straightforwardly and accurately, and self check is also 
made easy. In addition, by combination with related 
, equipment, it becomes easy to prepare editing sheets, as 
well as to carry out cuts for positives and negatives. 

5 O As for music composition, from the characteristics that 
videos have of being compact and easy to manipulate, as 
well as their easy adaptability to the development of 
musical equipment, it may expand the time and musical image 
creativity for pictures by permitting presentations 

10 including special effects. 

O As for dubbing, it shows high operability as permitting 
easier start, stop and repeating than conventional film 
projectors for video recorders, since its is possible to 
visualize in larger screens through video projectors and 

15 the like. In addition, through its use in association, 

it is possible to increase representation of details. 
O As for educational research, it is possible through the 
present method, to perform accurate analysis, frame by 
frame, especially for research and analysis of renowned 

20 movies and the like. In addition, film damage is obviously 
eliminated. 

O Software service and protection of copyrights 

Viewing the present method from the perspective of 
nowadays' imaging world, it can be said that it is a simple 

25 and straightforward closed system. By paying attention to 
this point, it becomes useful for the preservation in video 
form of companies' non-disclosed research material and the 
like. In addition, it can be considered as a prospective 
method in the sense of preservation and protection of 

30 copyrights of originals even for the so-called members-only 
services that has the primary purpose of preservation but also 
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have the characteristics of disclosure of their software to 
specific people or organizations, from film libraries that 
each country has to museums, public organizations and the like, 
(by utilizing cards with, secret codes, it is possible to 
5 pursue double copyrights protection and improvement of 
service quality.) 

O Through the present system, it becomes possible even for 
beginners to check 24-frame movie animations/celluloid 
pictures accurately and fast. 

10 As above mentioned, the staff work and their 

intercommunicability can be expanded by utilizing the present 
system' s method in an inexpensive way, even when looking back 
to nowadays' movie production world which has been reclusive 
in the backward sense of lacking enough funding and relying 

15 thoroughly on craftsmanship, constituting a rational, 
scientific and constructive new movie production method with 
educational impact. 

Patent applicant: Shigeo Hiyama 
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